A method for denoising a parallax map

Abstract

The application discloses a parallax map denoising method, comprising the following steps: step 1, acquiring the collected image of a binocular camera and calculating a parallax value to generate a parallax map; step 2, traversing parallax points in the parallax map according to a parallax threshold value, and dividing the parallax map into multiple layers according to the traversal result; step 3, counting the number of parallax points in each layer, and removing the layer whose number of parallax points is less than or equal to a corresponding plane threshold value; step 4, generating a denoised parallax map according to the remaining layers by using an image splicing method; wherein the values of the parallax threshold value and the plane threshold value are determined by the distance between a pixel point in the collected image and the binocular camera. According to the parallax threshold value and the plane threshold value, the application divides each parallax point in the parallax map into layers, realizes denoising filtering of the parallax map, and retains the discrete effective parallax points in the corresponding layer.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for denoising disparity maps. Background Technology

[0002] Binocular stereo vision technology is a technique that estimates disparity in depth maps of the same scene captured by binocular cameras to determine the depth of a 3D scene. The disparity maps obtained through disparity estimation algorithms often contain a large amount of noise, which significantly interferes with subsequent operations and analysis.

[0003] To remove these noise points, existing methods typically employ fixed-window filtering (window size is usually 11*9). This involves comparing the disparity values ​​of disparity points within the window with the disparity value of the center point, counting the number of disparity points with similar values, and if the requirement is not met, all disparity points within the window are considered noise and removed. However, this method has the following drawbacks:

[0004] Since the disparity map output by a binocular camera is semi-dense, there will be a relatively large number of disparity points (scattered points) with accurate disparity values ​​but scattered distribution. The number of these scattered points in a local area generally does not meet the requirements of existing window filtering methods, so a large number of these scattered points will be deleted, resulting in a reduction of effective disparity points in the disparity map.

[0005] In addition, some noise points that are relatively concentrated (have disparity values, but the disparity values ​​are not accurate) are retained because they meet the requirements of the window filtering method, resulting in poor disparity map filtering effect and increasing the difficulty of target recognition based on disparity maps.

[0006] CN115908170A discloses a method for denoising binocular images, comprising: calculating a first horizontal disparity matrix corresponding to the shallowest guiding image of the binocular image using a nonlocal mean filtering algorithm; calculating a first Euclidean distance between a first left-view guiding image and a first right-view guiding image in the first left-view guiding image using a nonlocal mean filtering algorithm based on the first horizontal disparity matrix; calculating a third Euclidean distance between the first left-view guiding image and a fourth Euclidean distance between the first right-view guiding image and the first left-view guiding image in the first left-view guiding image using a nonlocal mean filtering algorithm; and obtaining a denoised image of the corresponding binocular image based on the first, second, third, and fourth Euclidean distances, thereby achieving the effect of reducing image noise in the binocular image. However, this patent's method for image denoising is not comprehensive, the denoising effect is poor, and the method is relatively complex.

[0007] CN112381102A discloses an image denoising model generation method and an image denoising method. The method involves acquiring N first raw images with different Bayer arrays from the same scene, fusing these N first raw images to obtain corresponding N second raw images, establishing a sample image database based on the first and second raw images, and training a model to obtain an image denoising model. The raw image to be processed is then converted to a new format and input into the image denoising model to obtain a first denoised image. This first denoised image is then further converted to a new format to obtain the final denoised image. This patent achieves automatic image denoising based on the generated image denoising model, significantly reducing image noise, but the method is relatively complex.

[0008] Therefore, it is of great significance to propose a method that can effectively and comprehensively reduce noise on disparity maps while retaining discrete and effective disparity points on the disparity map. Summary of the Invention

[0009] The technical problem to be solved by this application is: how to perform noise reduction filtering on disparity maps while preserving discrete effective disparity points.

[0010] To solve the above-mentioned technical problems, this application provides the following technical solution:

[0011] A disparity map denoising method includes the following steps:

[0012] Step 1: Acquire images from the binocular camera and calculate the disparity value to generate a disparity map;

[0013] Step 2: Traverse the disparity points in the disparity map according to the disparity threshold, and divide the disparity map into multiple layers based on the traversal results;

[0014] Step 3: Count the number of disparity points in each layer and remove layers with a number of disparity points less than or equal to the corresponding plane threshold.

[0015] Step 4: Using image stitching, generate a denoised disparity map based on the remaining layers;

[0016] The values ​​of the disparity threshold and the plane threshold are determined by the distance between the pixels in the acquired image and the binocular camera.

[0017] In some embodiments, step 2 includes:

[0018] Step 2.1.1: Iterate through each disparity point in the disparity map in sequence, and determine the label of each disparity point based on the disparity threshold and the disparity value of each disparity point.

[0019] Step 2.1.2: Divide the disparity map into multiple layers according to the labels of each disparity point.

[0020] In some embodiments, step 2 includes:

[0021] Step 2.2.1: Determine the disparity threshold based on the disparity value of the current disparity point and the distance between the corresponding pixel and the stereo camera;

[0022] Step 2.2.2: Determine whether the disparity values ​​of the adjacent disparity points of the current disparity point are within the disparity threshold range.

[0023] If so, assign the current parallax point to the first layer containing the adjacent parallax points.

[0024] If not, the current parallax point will be classified into a second layer, which will be different from the first layer.

[0025] Among them, the adjacent disparity point is the upper adjacent disparity point and / or the left adjacent disparity point of the current disparity point.

[0026] In some embodiments, the disparity threshold includes left boundary disparity and right boundary disparity, where the left boundary disparity threshold is... L The calculation formula is:

[0027]

[0028] Right boundary parallax threshold R The calculation formula is:

[0029]

[0030] In the formula, X is the distance between the pixel and the stereo camera, D is the disparity value corresponding to the pixel, and δd is the set distance difference value.

[0031] In some embodiments, the adjacent disparity points are the upper adjacent disparity point and the left adjacent disparity point of the current disparity point, and the method further includes:

[0032] Determine the layer label;

[0033] Step 2.2.2 includes:

[0034] When the disparity values ​​of the upper adjacent disparity point and the left adjacent disparity point are both within the disparity threshold range, and the layer label of the layer containing the upper adjacent disparity point is different from the layer label of the layer containing the left adjacent disparity point,

[0035] Assign the current parallax point to the layer containing the left adjacent parallax point.

[0036] In some embodiments, after assigning the current disparity point to the layer containing the left adjacent disparity point, the method further includes:

[0037] Mark the layer labels of the layers containing adjacent parallax points and the layers containing the left adjacent parallax points;

[0038] After traversing all the disparity points in the disparity map, the corresponding layers are merged into one layer based on the labeling results of the layer labels, and the merged layer labels are generated.

[0039] In some embodiments, adjacent disparity points are:

[0040] In the disparity map, the upper adjacent disparity point above the current disparity point, whose disparity value is not 0, and / or,

[0041] The left-hand adjacent disparity point in the disparity map to the left of the current disparity point and whose disparity value is not 0.

[0042] In some embodiments, the disparity map denoising method further includes:

[0043] Step 5: Based on the denoised disparity map, map the right corrected image to a left image space to generate the left mapped image;

[0044] Step 6: According to the row and column coordinates in the left mapped image, count the number of pixels in the right corrected image corresponding to each pixel in the left mapped image. When the count is greater than 1, record the pixels in the multiple right corrected images corresponding to a pixel in the left mapped image as occluded pixels.

[0045] Step 7: Compare the disparity values ​​when the occluded pixel is mapped to the corresponding pixel in the left mapped image, retain the minimum disparity value, and update the remaining disparity values ​​to 0;

[0046] Step 8: Update the noise reduction parallax map.

[0047] In some embodiments, the formula for calculating the plane threshold is:

[0048]

[0049] In the formula, alpha is a preset constant, and X is the distance between the pixel and the stereo camera.

[0050] This application has the following beneficial effects:

[0051] (1) The disparity map denoising method provided in this application divides the disparity points in the disparity map into layers (regions) based on disparity threshold and plane threshold. Disparity points with large pixel intervals but close to the binocular camera can be treated as the same layer. Then, based on the number of disparity points in each layer, the layer with a smaller number of disparity points is deleted, thereby achieving denoising filtering of the disparity map and retaining the discrete effective disparity points in the corresponding layers, increasing the number of effective disparities and the density of the disparity map. Based on the denoised disparity map, the right corrected image is mapped to a left image space to generate a left mapped image. The row and column coordinates of the image are used to count the number of pixels in the right-corrected image corresponding to each pixel in the left-mapped image. When the count is greater than 1, the pixels in the right-corrected images corresponding to a pixel in the left-mapped image are recorded as occluded pixels. The disparity values ​​when the occluded pixels are mapped to the corresponding pixels in the left-mapped image are compared. The minimum disparity value is retained, and the remaining disparity values ​​are updated to 0 to obtain an updated denoised disparity map. Through the above method, the disparity map can be denoised and filtered while retaining discrete effective disparity points, and noise caused by occlusion can be effectively filtered out.

[0052] (2) The disparity map denoising method provided in this application can effectively and comprehensively denoise and filter out noise points on the disparity map, thereby greatly reducing the difficulty of target recognition based on the disparity map.

[0053] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0054] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0056] Figure 1 This is a flowchart illustrating a disparity map denoising method according to an embodiment of this application;

[0057] Figure 2 This is a schematic diagram of layer division for each disparity point on the disparity map in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of the mapping from the right corrected image to the left image space in an embodiment of this application;

[0059] Figure 4 These are images acquired in the same scene in the embodiments of this application;

[0060] Figure 5 This is a schematic diagram comparing the denoising effects of the fixed window filtering method and the denoising method of this application on disparity maps in the embodiments of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0062] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0063] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0064] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0065] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0066] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0067] Example 1

[0068] like Figure 1 As shown, this embodiment provides a disparity map denoising method, which includes the following steps:

[0069] Step 1: Acquire images from the binocular camera and calculate the disparity value to generate a disparity map;

[0070] Step 2: Traverse the disparity points in the disparity map according to the disparity threshold, and divide the disparity map into multiple layers based on the traversal results;

[0071] Step 3: Count the number of disparity points in each layer and remove layers with a number of disparity points less than or equal to the corresponding plane threshold.

[0072] Step 4: Using image stitching, generate a denoised disparity map based on the remaining layers;

[0073] The values ​​of the disparity threshold and the plane threshold are determined by the distance between the pixels in the acquired image and the binocular camera.

[0074] Specifically, this embodiment can output a reference image and a disparity image based on a binocular camera. The reference image is usually the image captured by the right camera in the binocular camera, also called the right correction image.

[0075] The disparity map consists of multiple disparity points, each corresponding to a disparity value. This disparity value can be used as the basis for target detection / matching. Therefore, it is necessary to filter the disparity map generated by the stereo camera to filter out disparity points with incorrect disparity values ​​caused by unclear image textures or occlusion.

[0076] This embodiment introduces the concept of layers. Based on the disparity threshold and the plane threshold, the disparity points in the disparity map are divided into layers (regions). According to the set disparity threshold, it can be determined whether two disparity points are located in the same layer and the corresponding layer label is determined.

[0077] In this embodiment, adjacent disparity points that meet the disparity threshold are regarded as a layer. If there is a hole between two disparity points (disparity value is 0) but the disparity values ​​of the two points are similar, that is, the disparity value of the previous disparity point is within the disparity threshold range determined by the current disparity point, the two disparity points are also regarded as being in the same layer.

[0078] This embodiment uses a line-by-line scanning method, from left to right and from top to bottom, to divide the layers based on the disparity value and disparity threshold of each disparity point.

[0079] This partitioning method allows disparity points with large pixel intervals but close proximity to the stereo camera to be grouped into a single layer. Then, based on the number of disparity points in each layer, layers with fewer disparity points are deleted. Noise reduction filtering is applied to the disparity map, retaining the scattered points in the corresponding layers. Finally, the remaining layers are stitched or composited using image stitching or image compositing techniques, ensuring the layer positions remain unchanged, to generate a denoised disparity map, increasing the number of effective disparities and the density of the disparity map.

[0080] It should be noted that if there is a region with a parallax value of 0 between adjacent layers, that region can be retained.

[0081] In some embodiments, the disparity threshold includes left boundary disparity and right boundary disparity, where the left boundary disparity threshold is... L The calculation formula is:

[0082]

[0083] Right boundary parallax threshold R The calculation formula is:

[0084]

[0085] In the formula, X is the distance between the pixel and the stereo camera, D is the disparity value corresponding to the pixel, and δd is the set distance difference value.

[0086] In some embodiments, the formula for calculating the plane threshold is:

[0087]

[0088] In the formula, alpha is a preset constant, and X is the distance between the pixel and the stereo camera.

[0089] Setting a plane threshold is used to determine whether a layer (region) is noisy, that is, whether there are too few effective disparity points in each region.

[0090] In some embodiments, to simplify the layer division, the labels of each disparity point can be directly determined based on the disparity threshold, and disparity points with the same label can be divided into a unified layer. Therefore, step 2 includes:

[0091] Step 2.1.1: Iterate through each disparity point in the disparity map in sequence, and determine the label of each disparity point based on the disparity threshold and the disparity value of each disparity point.

[0092] Step 2.1.2: Divide the disparity map into multiple layers according to the labels of each disparity point.

[0093] In some embodiments, to ensure the accuracy of layer segmentation, while retaining scatter points with true and valid disparity values ​​and filtering out noise points that are concentrated in distribution but have incorrect disparity values, this embodiment also provides a layer segmentation method to ensure the reliability and accuracy of disparity map noise reduction filtering. Therefore, step 2 includes:

[0094] Step 2.2.1: Determine the disparity threshold based on the disparity value of the current disparity point and the distance between the corresponding pixel and the stereo camera;

[0095] Step 2.2.2: Determine whether the disparity values ​​of the adjacent disparity points of the current disparity point are within the disparity threshold range. If yes, classify the current disparity point into the first layer where the adjacent disparity points are located; if no, classify the current disparity point into the second layer, which is different from the first layer. The adjacent disparity points are the upper adjacent disparity point and / or the left adjacent disparity point of the current disparity point.

[0096] In some embodiments, adjacent disparity points are: the upper adjacent disparity point in the disparity map above the current disparity point and whose disparity value is not 0, and / or,

[0097] The left-hand adjacent disparity point in the disparity map to the left of the current disparity point and whose disparity value is not 0.

[0098] Specifically, this embodiment uses a line-by-line scanning method, from left to right and from top to bottom, to divide the layers based on the disparity values ​​and disparity thresholds of each disparity point. The following explanation uses a disparity point in a certain region in the middle of the disparity map as an example.

[0099] like Figure 2As shown, for disparity points A and B in the same row, when traversing to disparity point B, even if there are many holes between A and B, if the disparity value of disparity point A is within the disparity threshold range of disparity point B, then the labels of disparity points A and B are set to be the same, and they can be treated as the same layer, denoted as Layer 1. However, when traversing to disparity point C, if the disparity value of disparity point B exceeds the disparity threshold corresponding to point C, then disparity point C is marked with a different label, treated as another layer, denoted as Layer 2. Then, the next disparity point is scanned to determine the label and layer of each disparity point.

[0100] Specifically, in this embodiment, each current disparity point with a value is compared with its nearest neighboring disparity points to its left and above (skipping holes and not considering the number of gap holes). If they are close (within the disparity threshold determined by the current disparity point), they are marked with the same label and grouped together with the holes (if any) between the two points with values ​​into a single layer. If they are not close, a new label is set for the current point, and it is treated as a new layer.

[0101] It should be noted that in this embodiment, the labels of the parallax points and the layer labels of the layers can be numbered 1, 2, ..., N.

[0102] like Figure 2 As shown, when traversing to disparity point D, it is determined whether its adjacent disparity point E and left adjacent disparity point F are within the disparity threshold determined by the current disparity point D, thereby determining the division of the layer.

[0103] If the upper adjacent disparity point E is within the disparity threshold determined by disparity point D, while the left adjacent disparity point F is not within the disparity threshold determined by disparity point D, then disparity point D is assigned to the layer (first layer) where the upper adjacent disparity point E is located; otherwise, disparity point D is assigned to the layer where the left adjacent disparity point F is located.

[0104] If neither the upper adjacent disparity point E nor the left adjacent disparity point F is within the disparity threshold determined by disparity point D, then disparity point D is treated as a new layer, i.e., the second layer.

[0105] If both the upper adjacent parallax point E and the left adjacent parallax point F are within the parallax threshold determined by parallax point D, it is necessary to determine whether the upper adjacent parallax point E and the left adjacent parallax point F are located on the same layer. If they are on the same layer, then parallax point D can be assigned to that layer. If they are on different layers, then the layers are divided according to the set division rules. For example, parallax point D can be assigned to the layer where the upper adjacent parallax point E is located, or parallax point D can be assigned to the layer where the left adjacent parallax point F is located, to ensure the accuracy of layer division.

[0106] In some embodiments, considering that the disparity value calculation process is performed row by row, for the case where both the upper adjacent disparity point E and the left adjacent disparity point F are within the disparity threshold determined by disparity point D, the adjacent disparity points are the upper adjacent disparity point and the left adjacent disparity point of the current disparity point. The layer division method further includes: determining the layer label of the layer; step 2.2.2 includes:

[0107] When the disparity values ​​of the upper adjacent disparity point and the left adjacent disparity point are both within the disparity threshold range, and the layer label of the layer containing the upper adjacent disparity point is different from the layer label of the layer containing the left adjacent disparity point, the current disparity point is assigned to the layer containing the left adjacent disparity point.

[0108] In some embodiments, after assigning the current disparity point to the layer containing the left adjacent disparity point, the method further includes: marking the layer labels of the layers containing the adjacent disparity points and the layer labels of the layers containing the left adjacent disparity points; after completing the traversal of the disparity points in the disparity map, the corresponding layers are merged into one layer according to the marking results of the layer labels, and the merged layer labels are generated.

[0109] Specifically, if the current disparity point is close to the adjacent disparity point to the left and close to the adjacent point above, but the disparity points to the left and above belong to two different layers, then these two different layers are marked and processed uniformly into one layer during post-processing after traversing the disparity points in the disparity map.

[0110] The above method helps to ensure the accuracy and reliability of layer division. While retaining the true and valid disparity values ​​of scattered points, it can filter out noise points that are concentrated in distribution but have incorrect disparity values.

[0111] In some embodiments, the acquired image is a right-corrected image, and the noise reduction method further includes:

[0112] Step 5: Based on the denoised disparity map, map the right corrected image to a left image space to generate the left mapped image;

[0113] Step 6: According to the row and column coordinates in the left mapped image, count the number of pixels in the right corrected image corresponding to each pixel in the left mapped image. When the count is greater than 1, record the pixels in the multiple right corrected images corresponding to a pixel in the left mapped image as occluded pixels.

[0114] Step 7: Compare the disparity values ​​when the occluded pixel is mapped to the corresponding pixel in the left mapped image, retain the minimum disparity value, and update the remaining disparity values ​​to 0;

[0115] Step 8: Update the noise reduction parallax map.

[0116] Specifically, if a pixel with coordinates (x, y) in the right corrected image has a disparity value of 100 in the disparity map, then the pixel coordinates mapped to the left mapped image are (x+100, y). During the process, it's possible for multiple pixels in the right corrected image to map to the same pixel in the left mapped image. In such cases, technical personnel analyze the disparity data to determine that this error is due to disparity matching errors caused by occlusion.

[0117] Therefore, after mapping the right-corrected image to a left image space, the number of right-image pixels corresponding to each point in the left space is counted sequentially according to the row and column coordinates in the left space.

[0118] When the number of pixels is greater than 1, retain the disparity value with the smallest disparity value used when mapping the pixels of the right corrected image, update the remaining disparity values ​​to 0, and generate a new disparity map.

[0119] Specifically, such as Figure 3 As shown, since the field of view of the left and right cameras in a binocular camera is not the same, there will be some visual occlusion areas in the images captured by the binocular camera for objects with overlapping positions.

[0120] For example, if object A1 is larger and located to the left front of object B1, when the binocular cameras are observing objects A1 and B1, from the perspective of the left camera (left lens), object A1 will cause some visual occlusion of object B1. The image of this visually occluded area can be displayed in the right-corrected image output by the right camera (right lens), as shown below. Figure 3 The shaded area in the image.

[0121] When the visual data processing chip outputs a disparity map, it uses confidence to determine the validity of all disparity points in the disparity map so as to output disparity points with obvious features. Therefore, even if a disparity point is in an occluded area, it will still be considered a valid disparity point and reflected in the output disparity map.

[0122] In the subsequent mapping process, if the features of the region corresponding to the disparity (incorrect disparity) of pixel E1 are relatively obvious, there is a possibility that pixel E1 in the occluded region may be matched to pixel D1 in the unoccluded region of the left image space.

[0123] In the subsequent pixel mapping process, pixel C1 in the unoccluded area will also be mapped to pixel D1. Therefore, we can conclude that the noise (point E1) caused by occlusion, although it has a correctly matched point in the disparity map, will have multiple disparity points in the right corrected image. That is, multiple points in the right corrected image are mapped to the same point in the left image space.

[0124] Since the visual data processing chip performs disparity matching based on the right image, the occluded area is located to the right of object A1. Therefore, it can be concluded that the disparity value of the occluded area is large.

[0125] Therefore, during the mapping process, if multiple points are found to be mapped to the same point, the point with the smaller disparity value is retained.

[0126] In this embodiment, the following explanation is provided to facilitate understanding of why a smaller disparity value is retained.

[0127] The rationale for retaining smaller parallax values ​​is as follows: Since the occlusion is present, it is easy to understand that the occluded area is located in the background area, so it is assumed that the pixels matched by the occluded area are also located in the background area.

[0128] First, since the parallax map uses the right corrected image as a reference, the occlusion area here refers to the occlusion area of ​​the right image relative to the left image (it exists in the right image but not in the left image). This type of area must be located to the left of the background area: (See diagram) Figure 3 Looking at it from above, object A1 is the foreground area and object B1 is the background area. It is easy to see that the occluded area is the left half of object B1.

[0129] Based on the assumption that the points matched in the occluded area are located in the background area, the correct matching pixels in the right image must also be located in the background area, as shown by point C1. Therefore, the erroneous parallax of the occluded area must be larger than the correct parallax, so we choose to retain the smaller parallax.

[0130] It should be noted that the above steps can achieve noise reduction filtering of the disparity map while retaining discrete effective disparity points, and can also effectively filter out noise caused by occlusion. Considering that the number of such disparity points is small and their impact on the density of the disparity map is also small, the occluded pixels can be directly deleted after they are counted, which helps to improve the disparity noise caused by repeated textures.

[0131] Using the above method, this embodiment calculates disparity values ​​for images acquired in the same scene, generates a disparity map, and then applies existing fixed-window filtering methods and the noise reduction method of this application to filter out noise points from the disparity map, comparing the noise reduction effects. The acquired images are attached. Figure 4 As shown in the attached figure, the noise reduction effects of the two methods are compared. Figure 5 As shown, where, Figure 5 The left image shows the noise reduction effect using the fixed window filtering method, and the right image shows the noise reduction effect using the noise reduction method of this application. The circled areas in the images represent concentrated noise points.

[0132] As attached Figure 5As shown in the upper left figure, when using the fixed window filtering method, concentrated noise points still exist on the disparity map, and a large number of discrete effective disparity points are deleted, resulting in poor filtering effect.

[0133] As attached Figure 5 As shown in the upper right figure, the noise reduction method of this application effectively filters out the concentrated noise points on the disparity map. Compared with the left figure, the right figure retains most of the discrete effective disparity points, making the data information on the disparity map rich and complete.

[0134] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of this application and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of this application should be included within the protection scope of this application. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for denoising a parallax map, the method comprising: The method comprises the following steps: Step 1, acquiring an image collected by a binocular camera and calculating a parallax value to generate a parallax map; Step 2, traversing parallax points in the parallax map according to a parallax threshold, and dividing the parallax map into multiple layers according to a traversal result; In the step 2, the method comprises: Step 2.1.1, traversing each parallax point in the parallax map in sequence, and determining a label of each parallax point according to the parallax threshold and a parallax value of each parallax point; Step 2.1.2, dividing the parallax map into multiple layers according to the label of each parallax point; Step 2.2.1, determining the parallax threshold according to the parallax value of a current parallax point and a distance between a corresponding pixel point and the binocular camera; Step 2.2.2, judging whether parallax values of adjacent parallax points of the current parallax point are within the parallax threshold range, if yes, dividing the current parallax point into a first layer in which the adjacent parallax points are located, if no, dividing the current parallax point into a second layer, the second layer being different from the first layer, wherein the adjacent parallax points are an upper adjacent parallax point and / or a left adjacent parallax point of the current parallax point; Step 3, counting a parallax point number of parallax points in each layer, and removing a layer in which the parallax point number is less than or equal to a corresponding plane threshold; Step 4, generating a denoising parallax map according to remaining layers in a manner of image stitching; wherein the parallax threshold and the plane threshold are determined by a distance between a pixel point in the collected image and the binocular camera.

2. The method of claim 1, wherein, The disparity threshold includes a left boundary disparity and a right boundary disparity, the left boundary disparity The calculation formula is: ， the right boundary disparity The calculation formula is: ， In the formula, D is the distance between the pixel point and the binocular camera, D is the parallax value corresponding to the pixel point, is the set distance difference value.

3. The method of claim 1, wherein, The adjacent parallax points are the upper adjacent parallax point and the left adjacent parallax point of the current parallax point, and the method further comprises: determining a layer label of the layer; In the step 2.2.2, the method comprises: when it is judged that the parallax values of the upper adjacent parallax point and the left adjacent parallax point are within the parallax threshold range, and the layer label of the layer in which the upper adjacent parallax point is located is different from the layer label of the layer in which the left adjacent parallax point is located, dividing the current parallax point into the layer in which the left adjacent parallax point is located.

4. The method of claim 3, wherein, After the current parallax point is divided into the layer in which the left adjacent parallax point is located, the method further comprises: marking the layer label of the layer in which the upper adjacent parallax point is located and the layer label of the layer in which the left adjacent parallax point is located; after traversing the parallax points in the parallax map, merging corresponding layers into one layer according to a marking result of the layer label, and generating a merged layer label.

5. The method of any one of claims 1-4, wherein, The adjacent parallax points are: the upper adjacent parallax point above the current parallax point in the parallax map and having a parallax value of 0, and / or the left adjacent parallax point left to the current parallax point in the parallax map and having a parallax value of 0.

6. The method of claim 1, wherein, The collected image is a right rectified image, and the method further comprises: Step 5, mapping the right rectified image to a left image space according to the denoising parallax map to generate a left mapping image; Step 6, the number of pixel points corresponding to each pixel point in the left mapping image in the right correction graph is counted in sequence according to the row and column coordinates in the left mapping image, and when the number is greater than 1, a plurality of pixel points in the right correction graph corresponding to a pixel point in the left mapping image are recorded as shielding pixel points; Step 7, the disparity values of the shielding pixel points when they are mapped to the corresponding pixel points in the left mapping image are compared, the minimum disparity value in the disparity values is retained, and the rest of the disparity values are updated to 0; Step 8, the denoising disparity map is updated.

7. The method of claim 1, wherein, The calculation formula of the plane threshold is: ， In the formula, alpha is a preset constant, is the distance between the pixel point and the binocular camera.

Images