Adaptive shadow area image dense matching cost construction method and system

Abstract

The present invention provides an adaptive shadow area image dense matching cost construction method and system, including inputting left and right epipolar line images, constructing a shadow detection model based on RGB color space angle and distance constraints; using the shadow detection model to perform shadow detection and marking on remote sensing images, including using the statistical sample expectation and variance principle to estimate the relative mean and variance of shadow points in the sampled shadow area, and then determining the range of possible values ​​of the remote sensing image shadow area in the RGB color space, to achieve preliminary shadow detection and marking; constructing a matching cost function based on the shadow detection and marking image, including constructing edge feature items that take into account the characteristics of the shadow area and the non-shadow area; calculating the disparity and generating a disparity map. The present invention can improve the correlation of the same-name image points in the shadow matching difficulty area, so that the disparity estimation can not only enhance the clarity of the edge of the object, but also reduce the matching holes in the shadow area, and significantly improve the quality of dense matching of the shadow-covered area image.

Description

Technical Field

[0001] The present invention belongs to the field of aerospace photogrammetry, and relates to a matching cost construction scheme for improving the quality of dense matching of images in areas where shadow matching is difficult, and in particular to a shadow detection and disparity map generation technology for remote sensing images. Background Art

[0002] Image dense matching has always been one of the hot research topics in the fields of photogrammetry, remote sensing and computer vision. After decades of development, today's image dense matching technology has both robust reliability and high matching accuracy and efficiency. The more mature and effective matching methods can be roughly divided into local dense matching and global dense matching. Local dense matching is further divided into adaptive window method, adaptive weight method and non-forward parallel method; global dense matching is divided into dynamic programming method, confidence propagation method and graph cut method. Generally speaking, local dense matching methods have high matching efficiency, but their matching accuracy is low, especially for weak texture areas of images; global dense matching methods have high matching accuracy, but the matching effect is poor for the edge areas of objects in complex scenes, and there are often problems of edge burrs and matching holes of objects, and they have poor real-time performance and low matching efficiency. Overall, today's dense matching technology still cannot meet the actual application needs for weak texture areas of images, especially large shadow areas or edge areas of objects.

[0003] Image shadow areas are difficult areas for dense matching and are widely present in urban scenes with complex building structures. When sunlight (usually considered to be parallel light under ideal conditions) at a certain moment shines on a high-rise building on the ground, large areas of shadows will be generated on the roof, wall and ground, including self-shadow, penumbra and umbra. Among them, the self-shadow on the wall of the high-rise building and the umbra projected on the ground seriously affect the correct expression of the edge information at the junction of the wall and the ground, and thus affect the correlation expression of the same-name image points of the stereo image pair, resulting in poor integrity and matching accuracy of the dense matching results of the stereo image.

[0004] Therefore, this technical field urgently needs to have a more practical high-quality dense matching technology solution that can fully take into account the characteristics of remote sensing images in shadow-occluded areas. Summary of the invention

[0005] In view of the shortcomings of existing disparity estimation methods, the present invention provides a matching cost construction technical solution which can significantly improve the quality of dense matching in difficult areas of image shadow matching.

[0006] The technical solution of the present invention is a method for constructing an adaptive shadow area image dense matching cost, comprising the following steps:

[0007] Step 1: Input the left and right epipolar images, build a shadow detection model based on the RGB color space angle and distance constraints, and geometrically describe the spatial distribution characteristics of the shadow area;

[0008] Step 2, using the shadow detection model to detect and mark the shadows of the remote sensing image, including using the shadow detection model constructed in step 1, using the statistical sample expectation and variance principle to estimate the relative mean and variance of the shadow points in the sampled shadow area, and then determining the range of possible values ​​of the shadow area of ​​the remote sensing image in the RGB color space, to achieve preliminary shadow detection and marking;

[0009] Step 3, constructing a matching cost function based on the shadow detection and marking image obtained in step 2, including taking the gradient truncated absolute difference function in the original cost function as a basis function, constructing an edge feature term that takes into account the features of the shadow area and the non-shadow area, and then combining it with the color term of the original cost function to realize the construction of the matching cost function;

[0010] Step 4: Calculate the disparity according to the matching cost function constructed in step 3 and generate a disparity map.

[0011] Moreover, in the shadow detection model based on angle and distance constraints of RGB color space, the shadow point information located on the ground and the wall is preliminarily obtained through the angle constraint condition, and the distance constraint condition is a re-constraint of the angle constraint condition. The final shadow point position is determined based on the distance constraint of the angle constraint.

[0012] Furthermore, in step 2, the point at pixel position (i, j) is constrained by angle to obtain the angle constrained color value X1(i, j) c , after the spatial distance constraint, the distance constraint color value X2(i,j) is obtained c , according to the value characteristics of the RGB space point, X1(i,j) c Further constraints are applied to obtain the RGB coordinates of the candidate shadow points, thereby obtaining the marked image Img with the candidate shadow area Cand_Shadow Then, according to the value characteristics of the RGB space point, X2(i,j) c Constraints are applied, and finally the RGB coordinates of the shadow point Shadow(i,j) are obtained, and the image Img with the marked shadow area is obtained. Shadow .

[0013] Moreover, in step 3, according to the shadow detection and marking image obtained in step 2, edge feature items that take into account the characteristics of the shadow area and the non-shadow area are constructed to enhance the correlation of the image to the same-name image points in the non-shadow area, and the edge expression ability of the pixels in the shadow area is made more sensitive to enhance the correlation of the same-name image points in the shadow area.

[0014] Moreover, in step 4, the AD-Census matching method is used to verify the transferability of the matching cost.

[0015] Moreover, it is used for shadow area matching of urban scene images with complex building structures.

[0016] On the other hand, an adaptive shadow area image dense matching cost construction system is provided, which is used to implement the adaptive shadow area image dense matching cost construction method as described above.

[0017] Furthermore, the following modules are included,

[0018] The first module is used to input the left and right epipolar images, build a shadow detection model based on the RGB color space angle and distance constraints, and geometrically describe the spatial distribution characteristics of the shadow area;

[0019] The second module is used to detect and mark shadows of remote sensing images using a shadow detection model, including estimating the relative mean and variance of shadow points in the sampled shadow area using the statistical sample expectation and variance principle based on the shadow detection model constructed in the first module, and then determining the range of possible values ​​of the shadow area of ​​the remote sensing image in the RGB color space, so as to achieve preliminary shadow detection and marking;

[0020] The third module is used to construct a matching cost function based on the shadow detection and marking image obtained by the second module, including taking the gradient truncated absolute difference function in the original cost function as the basis function, constructing an edge feature term that takes into account the characteristics of the shadow area and the non-shadow area, and then combining it with the color term of the original cost function to realize the construction of the matching cost function;

[0021] The fourth module is used to calculate the disparity and generate a disparity map according to the matching cost function constructed by the third module.

[0022] Alternatively, it includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the above-mentioned adaptive shadow area image dense matching cost construction method.

[0023] Alternatively, it comprises a readable storage medium having a computer program stored thereon, and when the computer program is executed, the adaptive shadow area image dense matching cost construction method as described above is implemented.

[0024] The present invention can improve the correlation of image points with the same name in the shadow matching difficulty area, so that the parallax estimation can not only enhance the clarity of the edge of the object, but also reduce the matching holes in the shadow area, and significantly improve the quality of dense matching of images in the shadow shielding area. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1This is a flowchart of creating a matching cost function according to an embodiment of the present invention.

[0026] Figure 2 It is a shadow detection model based on angle and distance constraints of RGB color space in an embodiment of the present invention.

[0027] Figure 3 This is a grayscale gradient range value mapping result diagram of an embodiment of the present invention.

[0028] Figure 4 : This is a graph showing the gradient comparison results of the original gradient term of the gradient truncated absolute difference function of the embodiment of the present invention and the edge feature term of the present invention, wherein part (a) is a local RGB image, part (b) is its corresponding grayscale image, and part (c) is the original function term Gradient map of, (d) part is Edge L (p) Gradient map.

[0029] Figure 5 3 is a comparative experimental effect diagram of the matching cost of an embodiment of the present invention before and after being migrated into the AD-Census matching method, wherein part (a) is the left nuclear line image, part (b) is the shadow detection and marking result of the present invention, part (c) is the disparity reference value converted from the LiDAR point cloud, part (d) is the disparity map of the original method AD_Census, and part (e) is the AD_Census disparity map after migrating the cost of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0031] In view of the problem that shadow occlusion seriously restricts the quality of image dense matching, the present invention provides a method for constructing image dense matching cost in difficult areas of shadow matching based on edge feature constraints. This method uses shadow detection results as the basis for constructing a matching cost function. First, a shadow detection model based on angle and distance constraints in RGB color space is established to detect and mark shadow areas of different scenes; then an edge feature item is constructed and introduced into the matching cost calculation to form an image dense matching cost function that can adapt to difficult areas of shadow matching. In this embodiment, the Toronto public data set is used as an example to verify the transferability of the cost function and image dense matching. The data set contains data such as the original RGB image and the LiDAR point cloud corresponding to the survey area.

[0032] In specific implementation, the technical solution of the present invention can be automatically operated by computer programming. In the embodiment, shadow detection and marking are preferably implemented by MATLAB programming, and calculation of matching cost and dense matching are preferably implemented by Visual C++ programming.

[0033] The adaptive shadow area image dense matching method provided by the embodiment of the present invention can be referred to Figure 1 , including the following corrections:

[0034] Step 1: Input the left and right epipolar images and build a shadow detection model based on the angle and distance constraints of the RGB color space.

[0035] In urban scenes, the wall surface materials of buildings are mainly bricks, and the ground materials mainly include asphalt, cement and vegetation. In the same remote sensing image or the same urban scene, the ground with relatively consistent color effects can be seen. This scene feature makes the shadow area pixels located on the ground always present a certain spatial distribution in the RGB color space. To this end, the present invention constructs a shadow detection model based on the angle and distance constraints of the RGB color space to geometrically describe the spatial distribution characteristics of the shadow area.

[0036] The present invention further proposes a shadow detection model based on the angle and distance constraints of the RGB color space. The angle constraint condition can preliminarily obtain the shadow point information located on the ground and the wall, and the distance constraint condition is a re-constraint of the angle constraint condition. The distance constraint based on the angle constraint can determine the final shadow point position.

[0037] See also Figure 2 , an embodiment establishes a shadow detection model based on angle and distance constraints of RGB color space. Figure 2 In , the three spatial planes P1, P2, and P3 respectively represent the planes formed by the spatial points when the blue channel value B is equal to the green channel value G, the red channel value R is equal to the blue channel value B, and the red channel value R is equal to the green channel value G in the RGB color space; the vector The straight line where is located is the grayscale line formed by the spatial points of R=G=B, and is also the intersection of the three planes P1, P2 and P3; the vector for Any direction vector; point s is a pixel in any shadow area, and the corresponding coordinate vector in RGB color space is Where r, g, and b are the red channel values, green channel values, and blue channel values ​​corresponding to point s. Represents the vectors in the three coordinate axes corresponding to point s in the RGB color space; ф(s) is a vector and The angle of the plane.

[0038] The spatial position of any image point s is first constrained by the spatial angle ф(s), that is, the vector and and The angle ф(s) of the plane can be expressed as:

[0039]

[0040] In the formula, point s represents any point in the shaded area S. area The image point inside, || || represents the vector length;

[0041] vector Length Available vectors The length of the corresponding vector and To determine:

[0042]

[0043] Then for the shadow area S area Any image point s in the space is constrained by the spatial distance Dist(s). The spatial relative distance Dist(s) can be calculated based on the vector With direction vector To determine:

[0044]

[0045] The direction vector in the formula and vector The corresponding vector length can be written as:

[0046]

[0047]

[0048] Where n∈(0,255] and n∈Ν + , n is a positive integer set N + Any integer in .

[0049] Therefore, Dist(s) can be further written as:

[0050]

[0051] The position Loc(Dist(s),ф(s)) of the RGB color space of the image point s can be determined as:

[0052]

[0053] Step 2: Use the shadow detection model to detect and mark shadows in remote sensing images.

[0054] The present invention determines the spatial position of the shadow points given in step 1 based on the model, and adopts the statistical sample expectation and variance principle to estimate the relative mean and variance of the shadow points in the sampled shadow area, and then determines the possible value range of the shadow area of ​​the remote sensing image in the RGB color space, so as to realize preliminary shadow detection and marking.

[0055] Since the statistical sample expectation and variance concepts are used to realize the angle constraint of shadow points, and thereby realize shadow detection and marking, the shadow detection capability of the shadow detection model has more than a single adaptive capability, that is, the shadow detection model constructed based on statistical concepts in the present invention can be applied to shadow detection and marking in different urban scenes.

[0056] The embodiment determines the model (6) for shadow detection and marking based on the spatial position of the shadow point Loc(Dist(s),ф(s)) in step 1.

[0057] When sampling the shadow area S spl_area Any shadow point s within spl The spatial angle value is equal to ф(s spl ), the corresponding numerical frequency prec(ф(s spl ))for:

[0058]

[0059] In the formula, ф(s spl ) is the sampling shadow point s spl The spatial angle value, Num(ф(s spl )) is the sampling area S spl_area The spatial angle inside is equal to ф(s spl ) of all pixels; Sum(s spl ) is the sampling shadow area S spl_area The number of all pixels within.

[0060] ф(s spl ) can be expressed as

[0061]

[0062] In the formula, the set Ω refers to the sampling shadow area S spl_area Inner space angle ф(s spl ) is a set of possible values. m (s spl ) indicates ф(s spl ) value is m, perc(ф m (s spl )) represents the corresponding numerical frequency.

[0063] ф(sSpl ) It can be expressed as:

[0064]

[0065] Based on the above and Parameters can be used to detect and mark shadows on the input image Img. If the size of the image Img is set to rows×cols pixels (rows is the number of rows, cols is the number of columns), the point at the pixel position (i, j) can obtain the angle-constrained color value X1(i, j) after the angle constraint. c , after the spatial distance constraint, the distance constraint color value X2(i,j) can be obtained c :

[0066]

[0067] Where (i, j) is the index position of any pixel in Img, i is the row number where the pixel is located and i≤rows, j is the column number where the pixel is located and j≤cols. The spatial angle corresponding to the pixel is ф(s(i, j)), the corresponding distance is Dist(s(i, j)), c∈{R,G,B}, as in img R (i, j) is the R value of the pixel Img(i, j) in the input image in the RGB color space.

[0068] Next, X1(i,j) is converted into c Further constraints can be obtained to obtain the RGB coordinates of the candidate shadow point Cand_Shadow(i,j):

[0069] Cand_Shadow(i,j)=X1(i,j) c ,X1(i,j) c <255 (11)

[0070] The marked image Img with candidate shadow areas can be obtained Cand_Shadow Similarly, according to the value characteristics of the RGB space point, X2(i,j) c Constraints are applied and the RGB coordinates of the shadow point Shadow(i,j) are finally obtained:

[0071] Shadow(i,j)=X2(i,j) c ,X2(i,j) c <255 (12)

[0072] Finally, the marked image Img with shadow area can be obtained. Shadow .

[0073] Step 3: Construct a matching cost function based on the shadow detection and marking results of step 2.

[0074] Taking into account the universality of the correlation expression of the marked stereo image pairs, the present invention uses the widely used gradient truncated absolute difference function as the basis function (original cost function), focuses on constructing edge feature terms that take into account the characteristics of the shadow area and the non-shadow area, and then combines it with the color term of the original cost function to realize the construction of the matching cost function.

[0075] In step 3, edge feature items based on the characteristics of the shadow area and the non-shadow area are constructed according to the shadow detection and marking images obtained in step 2. This can not only enhance the correlation of the same-name image points in the non-shadow area of ​​the stereo image pair, but also improve the sensitivity of the pixel edge expression in the shadow area, greatly enhancing the correlation of the same-name image points in the shadow area.

[0076] In the embodiment, the left image I L The pixel point p in the right image I R The matching cost C between the pixel points pd in d (p) can be expressed as:

[0077] C d (p) = (1-α) × min (|I L (p)-I R (pd)|,τ Col )+αmin(|Edge L (p)-Edge R (pd)|,τ Edge )(13)

[0078] In the formula, || is the absolute value symbol; I L (p) and I R (pd) is the left image I L The pixel point p in the right image I R The color item of the pixel point pd in Edge L (p) and Edge R (pd) is the left image I L The pixel point p in the right image I R The edge feature item of the pixel point pd in; d is the disparity; α is the factor that balances the weights of the color item and the edge item; τ Col With τ Edge is a cutoff value to reduce the influence of the blocked pixels. Col With τ Edge The preferred values ​​are 0.11, 7 and 2 respectively. In the specific implementation, other suitable values ​​can be adopted according to experiments.

[0079] The present invention creatively proposes that Edge in formula (13) L The term (p) replaces the original cost function Here, the edge feature term Edge L (p) can not only refine the correlation expression of the left and right images to the edge of the non-shadow area, but also introduce the shadow feature expression function for the shadow area marked in step 2, so as to strengthen the correlation expression of the left and right images to the same-name image points in the shadow area. L (p) is used as an example to illustrate the calculation.

[0080] For any pixel p in the marked image, the modulus Mag of its gradient vector can be calculated L (p) and direction Mag D (p):

[0081]

[0082] Among them, g L_x (p) and g L_y (p) represents the x and y components of the grayscale value gradient of pixel p in the left epipolar image.

[0083] Considering that the original image is disturbed by factors other than its own intensity, such as environmental radiation and the relative position between the camera and the object, the gradient expression of some pixels in the image will be disturbed and the edge gradient information cannot be correctly expressed. In order to enhance the sensitivity of the edge feature items of the object, in addition to considering the x and y components of the gray value gradient, the gradient direction Mag D The value range of (p) is given by Map to [0,2π] to obtain the gradient direction Dir after the image point p is mapped L (p) (in radians) (see Figure 3 , taking the first quadrant as an example, Dir L The (p) value is determined by the x and y components of the grayscale gradient of the left nuclear line image (corresponding to g L_x (p) and g L_y (p)), can be obtained by calculating the inverse tangent function arctan, that is, Dir L (p) = arctan(g L_y (p) / g L_x (p)), the other three quadrants are similar, the second, third, and fourth quadrants are Dir L (p) = arctan(g L_y (p) / g L_x (p))+π、Dir L (p) = arctan(g L_y (p) / g L_x(p))+π、Dir L (p) = arctan(g L_y (p) / g L_x (p))+2π.

[0084] Finally, the marked shadow area and non-shadow area are subjected to the edge feature function Edge L For the construction of (p), see formula (15).

[0085]

[0086] In the formula, and Display image The direction and magnitude of the gradient vector of the image point p; and Display image The x and y components of the grayscale value gradient of the image point p; and Display image The x and y components of the grayscale gradient, g L_x (p) and g L_y (p) represents the x and y components of the grayscale value gradient of the pixel p in the input image Img.

[0087] Based on the Edge in the cost function of the present invention L (p) The feature items can be obtained Figure 4 The edge expression effect. Figure 4 In the figure, (a) is the local RGB image, (b) is the corresponding grayscale image, and (c) is the original function term. Gradient map of , (d) is Edge L (p) Gradient map. Compare Figure 4 (c) and (d) show that the edge feature function Edge L (p) Compared with the original function More detailed information about the edge of objects can be obtained, especially the edge information of objects marked by the red box and located in the shadow area.

[0088] Similarly, according to formula (15), the edge feature function Edge of the pd pixel on the right image can be obtained: R (pd).

[0089] Step 4: Calculate the disparity and perform image dense matching based on the matching cost function constructed in step 3.

[0090] In order to obtain high-quality dense matching results, a transferability experiment is conducted on the matching cost function constructed in step 3. The present invention selects the AD-Census matching method that takes into account both matching efficiency and effect to verify the transferability of the matching cost, and analyzes the effect of the disparity map through comparative experiments.

[0091] In step 4, the complete implementation of the matching cost implementation example requires that the cost has very good transferability, and the smooth implementation of the cost in AD-census can prove that this method has good transferability. At the same time, it can also overcome the mismatching problem of the original AD-census method in the difficult area of ​​shadow matching, and to a certain extent improve the accuracy of the image dense matching method.

[0092] In the embodiment, the matching cost function constructed in step 3 is transferred to the AD-Census method, and the disparity maps before and after the transfer are as follows: Figure 5 As shown. Figure 5 In the figure, several areas with different building structures are randomly selected for matching results display. The first column is a partial enlarged image of a non-building dense area. The building geometry is relatively complex. In addition to buildings with straight edge structures, there are buildings with curved edge structures and buildings with sawtooth edge structures. The roofs include both irregular sloping roofs and irregular ancillary buildings, and the shadows are obvious. The second column is a group of high-rise buildings with relatively regular geometric structures. Most of them are horizontal roofs with ancillary buildings. The relative position relationship between the buildings is relatively diverse. Some areas are accompanied by mirror reflections, and the shadows are obvious. The third column is a group of buildings with irregular geometric structures. The roof structures are rich and diverse, and the shadows are obvious. Figure 5 (a) is the left nuclear line image, Figure 5 (b) is the shadow detection and marking result of the present invention, Figure 5 (c) is the disparity reference value converted from LiDAR point cloud, Figure 5 (d) is the disparity map of the original method AD_Census, Figure 5 (e) is the AD_Census disparity map after migrating the cost of the present invention.

[0093] analyze Figure 5 It can be found that:

[0094] 1) Compared with the non-building dense area with complex building geometric structure characteristics (taking the first column as an example), the method embedded with the cost of the present invention not only improves the outer expansion problem of the edge of the object in the shadow area (the area marked by the red box) of the original AD_Census method to a certain extent, but also improves the parallax redundancy problem in the shadow area between the gaps of the building walls;

[0095] 2) Compared with the building-dense area (taking the second column as an example), the AD_Census method after embedding the cost of the present invention greatly improves the parallax expansion problem of the shadow area between adjacent buildings in the original AD_Census method (the area marked by the red box);

[0096] 3) Compared with the building group with irregular geometric structure characteristics (taking the third column as an example), the AD_Census method after embedding the cost of the present invention improves the edge expansion and parallax adhesion problems that are prone to occur in the shadow area of ​​the original AD_Census method to a certain extent (the area marked by the red box).

[0097] In summary, the matching cost construction method proposed in the present invention not only has good transferability, but also overcomes the mismatching problem of the original scheme in the difficult shadow matching area for the shadow area between adjacent buildings in a densely populated area with complex geometric structures or the shadow area of ​​the roof. It not only enhances the clarity of the edge of the object, but also reduces the dense matching holes in the shadow-shaded area, and significantly improves the quality of dense matching of images in the shadow-shaded area.

[0098] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0099] In some possible embodiments, an adaptive shadow area image dense matching cost construction system is provided, comprising the following modules:

[0100] The first module is used to input the left and right epipolar images, build a shadow detection model based on the RGB color space angle and distance constraints, and geometrically describe the spatial distribution characteristics of the shadow area;

[0101] The second module is used to detect and mark shadows of remote sensing images using a shadow detection model, including estimating the relative mean and variance of shadow points in the sampled shadow area using the statistical sample expectation and variance principle based on the shadow detection model constructed in the first module, and then determining the range of possible values ​​of the shadow area of ​​the remote sensing image in the RGB color space, so as to achieve preliminary shadow detection and marking;

[0102] The third module is used to construct a matching cost function based on the shadow detection and marking image obtained by the second module, including taking the gradient truncated absolute difference function in the original cost function as the basis function, constructing an edge feature term that takes into account the characteristics of the shadow area and the non-shadow area, and then combining it with the color term of the original cost function to realize the construction of the matching cost function;

[0103] The fourth module is used to calculate the disparity and generate a disparity map according to the matching cost function constructed by the third module.

[0104] In some possible embodiments, an adaptive shadow area image dense matching cost construction system is provided, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute an adaptive shadow area image dense matching cost construction method as described above.

[0105] In some possible embodiments, an adaptive shadow area image dense matching cost construction system is provided, including a readable storage medium, on which a computer program is stored. When the computer program is executed, an adaptive shadow area image dense matching cost construction method as described above is implemented.

[0106] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. An adaptive shadow image dense matching cost construction method, characterized by: The following steps are involved: Step 1: Input the left and right epipolar images, build a shadow detection model based on the angle and distance constraints of the RGB color space, and geometrically describe the spatial distribution characteristics of the shadow area; The shadow detection model based on the angle and distance constraints of the RGB color space is constructed in such a way that three spatial planes P1, P2, and P3 are set to represent the planes formed by the spatial points when the blue channel value B is equal to the green channel value G, the red channel value R is equal to the blue channel value B, and the red channel value R is equal to the green channel value G in the RGB color space respectively; vector The straight line where is located is the grayscale line formed by the spatial points of R=G=B, and is also the intersection of the three planes P1, P2 and P3; the vector for Any direction vector; point s is a pixel in any shadow area, and the corresponding coordinate vector in RGB color space is Where r, g, and b are the red channel values, green channel values, and blue channel values ​​corresponding to point s, respectively. Let the vector Represents the vectors in the three coordinate axes corresponding to point s in the RGB color space; ф(s) is a vector and The angle of the plane; The spatial position of any image point s is first constrained by the spatial angle ф(s). The vector and and The angle ф(s) of the plane is expressed as: In the formula, point s represents any point in the shaded area S. area The image point inside, |||| represents the vector length; vector Length Use vector The length of the corresponding vector and Sure: Then for the shadow area S area Any image point s in the space is constrained by the spatial distance Dist(s), and the spatial relative distance Dist(s) is calculated according to the vector With direction vector Sure: The direction vector in the formula and vector The corresponding vector length is written as: Where n∈(0,255] and n∈Ν + , n is a positive integer set N + Any integer in ; Dist(s) is further written as: The position Loc(Dist(s),ф(s)) of the RGB color space of the image point s is determined as: Step 2, using the shadow detection model to detect and mark the shadows of the remote sensing image, including using the shadow detection model constructed in step 1, using the statistical sample expectation and variance principle to estimate the relative mean and variance of the shadow points in the sampled shadow area, and then determining the range of possible values ​​of the shadow area of ​​the remote sensing image in the RGB color space, to achieve preliminary shadow detection and marking; Step 3, constructing a matching cost function based on the shadow detection and marking image obtained in step 2, including taking the gradient truncated absolute difference function in the original cost function as the basis function, constructing an edge feature term that takes into account the features of the shadow area and the non-shadow area, and then combining it with the color term of the original cost function to realize the construction of the matching cost function; the implementation method is as follows, Left Image I L The pixel point p in the right image I R The matching cost C between the pixel points pd in d (p) is expressed as: C d (p)=(1-α)×min(|I L (p)-I R (p-d)|,τ Col )+αmin(|Edge L (p)-Edge R (p-d)|,τ Edge ) In the formula, || is the absolute value symbol; I L (p) and I R (pd) is the left image I L The pixel point p in the right image I R The color item of the pixel pd in Edge L (p) and Edge R (pd) is the left image I L The pixel point p in the right image I R The edge feature item of the pixel point pd in; d is the disparity; α is the factor that balances the weights of the color item and the edge item; τ Col With τ Edge is the cutoff value; For any pixel p in the left image, the edge feature function Edge L (p) is calculated as follows, For any pixel p in the marked image, calculate the modulus Mag of its gradient vector L (p) and direction Mag D (p): Among them, g L_x (p) and g L_y (p) represents the x and y components of the grayscale gradient of pixel p in the left epipolar image; The gradient direction Mag D The value range of (p) is given by Map to [0,2π] to obtain the gradient direction Dir after the image point p is mapped L (p); Finally, the marked shadow area and non-shadow area are subjected to the edge feature function Edge L (p) construction, In the formula, and Display image The direction and magnitude of the gradient vector of the image point p; and Display image The x and y components of the grayscale value gradient of the image point p; and Display image The x and y components of the grayscale gradient, g L_x (p) and g L_y (p) represents the x and y components of the grayscale value gradient of the pixel p in the input image Img; Similarly, the edge feature function Edge of the pd pixel on the right image is obtained according to the above formula: R (pd); Step 4: Calculate the disparity according to the matching cost function constructed in step 3 and generate a disparity map.

2. The method for constructing the adaptive shadow area image dense matching cost according to claim 1, characterized in that: In the shadow detection model based on angle and distance constraints of RGB color space, the shadow point information on the ground and the wall is preliminarily obtained through the angle constraint condition, and the distance constraint condition is a re-constraint of the angle constraint condition. The final shadow point position is determined based on the distance constraint of the angle constraint.

3. The method for constructing the adaptive shadow area image dense matching cost according to claim 1, characterized in that: In step 2, the point at pixel position (i, j) is constrained by angle to obtain the angle constrained color value X1(i, j) c , after the spatial distance constraint, the distance constraint color value X2(i,j) is obtained c , according to the value characteristics of the RGB space point, X1(i,j) c Further constraints are applied to obtain the RGB coordinates of the candidate shadow points, thereby obtaining the marked image Img with the candidate shadow area Cand_Shadow Then, according to the value characteristics of the RGB space point, X2(i,j) c Constraints are applied, and finally the RGB coordinates of the shadow point Shadow(i,j) are obtained, and the image Img with the marked shadow area is obtained. Shadow .

4. The method for constructing the adaptive shadow area image dense matching cost according to claim 1, characterized in that: In step 3, according to the shadow detection and marking image obtained in step 2, edge feature items taking into account the features of the shadow area and the non-shadow area are constructed to enhance the correlation of the image to the same-name image points in the non-shadow area, and the edge expression ability of the pixels in the shadow area is made more sensitive to enhance the correlation of the same-name image points in the shadow area.

5. The method for constructing the adaptive shadow area image dense matching cost according to claim 1, characterized in that: In step 4, the AD-Census matching method is used to verify the transferability of the matching cost.

6. The method for constructing an adaptive shadow area image dense matching cost according to any one of claims 1 to 5, characterized in that: Used for shadow area matching in urban scene images with complex building structures.

7. An adaptive shadow area image dense matching cost construction system, characterized by: Used to implement an adaptive shadow area image dense matching cost construction method as described in any one of claims 1-6.

8. The adaptive shadow area image dense matching cost construction system according to claim 7, characterized in that: Includes the following modules, The first module is used to input the left and right epipolar images, build a shadow detection model based on the RGB color space angle and distance constraints, and geometrically describe the spatial distribution characteristics of the shadow area; The second module is used to detect and mark shadows of remote sensing images using a shadow detection model, including estimating the relative mean and variance of shadow points in the sampled shadow area using the statistical sample expectation and variance principle based on the shadow detection model constructed in the first module, and then determining the range of possible values ​​of the shadow area of ​​the remote sensing image in the RGB color space, so as to achieve preliminary shadow detection and marking; The third module is used to construct a matching cost function based on the shadow detection and marking image obtained by the second module, including taking the gradient truncated absolute difference function in the original cost function as the basis function, constructing an edge feature term that takes into account the characteristics of the shadow area and the non-shadow area, and then combining it with the color term of the original cost function to realize the construction of the matching cost function; The fourth module is used to calculate the disparity and generate a disparity map according to the matching cost function constructed by the third module.

9. The adaptive shadow area image dense matching cost construction system according to claim 7, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute an adaptive shadow area image dense matching cost construction method as described in any one of claims 1-6.

10. The adaptive shadow area image dense matching cost construction system according to claim 7, characterized in that: It includes a readable storage medium, on which a computer program is stored. When the computer program is executed, an adaptive shadow area image dense matching cost construction method as described in any one of claims 1 to 6 is implemented.

Images