High-resolution remote sensing image forest three-dimensional reconstruction method and system

Abstract

The application provides a high-resolution remote sensing image forest three-dimensional reconstruction method and system, generates a forest parallax map of a target forest region through a forest remote sensing image pair of the target forest; extracts matching abnormal pixel points in remote sensing image matching through a parallax mapping relationship of pixel points in the forest parallax map; extracts an independent closed patch with a crown topological structure as a boundary from the forest remote sensing image of the target forest region; constructs a multi-level concentric ring search region for the independent closed patch where the matching abnormal pixel points are located; determines parallax correction parameters of the matching abnormal pixel points according to effective matching pixel points in each concentric ring search region, obtains all corrected parallax points of the target forest region according to the parallax correction parameters, and reconstructs a forest three-dimensional point cloud model of the target forest region based on all the corrected parallax points. The technical scheme provided by the application can overcome the destructive influence of matching abnormal points caused by the complex structure of the forest canopy on the three-dimensional reconstruction precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and more particularly, to a high-resolution remote sensing image forest three-dimensional reconstruction method and system. BACKGROUND

[0002] Remote sensing image processing is a key technology for obtaining, analyzing and interpreting information of the earth's surface and atmosphere. In recent years, the popularity of high-resolution satellites (such as sub-meter optical satellites) and synthetic aperture radar (SAR) has led to a huge demand for data processing, driving the iteration of core technologies such as radiometric correction, geometric precision correction, and image fusion. At the same time, the introduction of artificial intelligence and deep learning has greatly improved the accuracy of image classification and target detection, providing key technical support for environmental monitoring, urban planning, disaster emergency response and other fields, forming a complete technology chain from data acquisition to intelligent interpretation.

[0003] In existing remote sensing image processing, remote sensing image processing aims to eliminate data errors, enhance effective information, and extract thematic features. First, radiometric correction is used to eliminate sensor response bias, atmospheric scattering and other interference, and convert image gray values to real object reflectivity. Geometric correction is based on ground control points, which corrects geometric distortion caused by terrain undulations and sensor attitude through coordinate conversion, achieving accurate matching of images and geographic coordinates. However, in high-resolution remote sensing image forest three-dimensional reconstruction, the target forest area often has problems of mutual occlusion of forest canopy leaves and high-frequency texture characteristics. Mutual occlusion of forest canopy leaves and high-frequency texture characteristics can cause non-continuous disparity jumps (i.e., matching abnormalities) during image matching, thereby reducing the geometric accuracy of the forest three-dimensional point cloud model. Therefore, how to overcome the destructive impact of matching abnormal points caused by the complex structure of forest canopy on the accuracy of three-dimensional reconstruction has become a difficult problem in the industry. SUMMARY

[0004] The present application provides a high-resolution remote sensing image forest three-dimensional reconstruction method and system, which can overcome the destructive impact of matching abnormal points caused by the complex structure of forest canopy on the accuracy of three-dimensional reconstruction.

[0005] In a first aspect, the present application provides a high-resolution remote sensing image forest three-dimensional reconstruction method, comprising the following steps:

[0006] Obtaining a pair of forest remote sensing images of a target forest area, performing pixel-level disparity analysis on the pair of forest remote sensing images, and generating a forest disparity map of the target forest area;

[0007] Locating matching abnormal pixels during remote sensing image matching from the pair of forest remote sensing images through the disparity mapping relationship of the pixels in the forest disparity map;

[0008] based on the edge profile continuity of the forest canopy, performing patch segmentation on the forest remote sensing image of the target forest region to obtain independent closed patches with the tree crown topological structure as the boundary;

[0009] For the independent closed patch where the matching abnormal pixel point is located, a multi-level concentric ring search region is constructed with the matching abnormal pixel point as the center in the independent closed patch, wherein the maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumscribed circle of the independent closed patch.

[0010] According to the effective matching pixel points in each level of the concentric ring search region, the disparity correction parameters of the matching abnormal pixel point in the remote sensing image matching are determined, and the matching abnormal pixel point is corrected according to the disparity correction parameters, and then all the corrected disparity points of the target forest region are obtained. The forest three-dimensional point cloud model of the target forest region is reconstructed based on all the corrected disparity points.

[0011] In some embodiments, the pixel-level disparity analysis on the forest remote sensing image pair is performed to generate the forest disparity map of the target forest region, specifically including:

[0012] Performing geometric correction on the forest remote sensing image pair to obtain a corrected forest remote sensing image pair;

[0013] Extracting corresponding feature point pairs from the corrected forest remote sensing image pair to form a corresponding feature point pair set;

[0014] Calculating the disparity value of the corresponding pixel points for each corresponding feature point pair in the corresponding feature point pair set to obtain all the disparity values;

[0015] According to all the disparity values, a forest disparity map of the target forest region is constructed.

[0016] In some embodiments, the matching abnormal pixel point in the remote sensing image matching is located from the forest remote sensing image pair through the disparity mapping relationship of the pixel points in the forest disparity map, specifically including:

[0017] Determining the disparity threshold range in the remote sensing image matching process based on the disparity mapping relationship of the pixel points in the forest disparity map;

[0018] Screening the pixel points with disparity values exceeding the disparity threshold range from the forest disparity map as candidate abnormal points;

[0019] Taking the pixel points corresponding to the candidate abnormal points in the forest remote sensing image pair as the matching abnormal pixel points in the remote sensing image matching.

[0020] In some embodiments, based on the edge profile continuity of the forest canopy, performing patch segmentation on the forest remote sensing image of the target forest region to obtain independent closed patches with the tree crown topological structure as the boundary specifically includes:

[0021] obtain a forest remote sensing image of a target forest area, and perform Gaussian filtering denoising on the forest remote sensing image to obtain a smoothed forest remote sensing image;

[0022] extract a crown layer edge profile map from the smoothed forest remote sensing image;

[0023] perform a morphological connection operation on the crown layer edge profile map to obtain a continuous crown layer edge profile of a forest crown layer;

[0024] based on the continuous crown layer edge profile, perform initial segmentation on the forest remote sensing image using a region growing algorithm to obtain initial patches;

[0025] perform morphological correction on the initial patches according to tree crown topological structure characteristics to obtain independent closed patches with tree crown topological structure as a boundary.

[0026] In some embodiments, for an independent closed patch in which the matching abnormal pixel point is located, a multi-level concentric ring search region is constructed with the matching abnormal pixel point as the center in the independent closed patch, specifically including:

[0027] determine the minimum circumscribed circle radius of the independent closed patch in which the matching abnormal pixel point is located;

[0028] use the minimum circumscribed circle radius as the maximum search radius of the multi-level concentric ring search region, and set the radius increment step of the concentric rings;

[0029] starting from an initial radius, construct each level of concentric ring in turn according to the radius increment step with the matching abnormal pixel point as the center until the maximum search radius is reached;

[0030] cut each level of concentric ring according to the boundary of the independent closed patch to form a multi-level concentric ring search region in the independent closed patch.

[0031] In some embodiments, determining the parallax correction parameter of the matching abnormal pixel point when matching the remote sensing image according to the valid matching pixel points in each level of concentric ring search region specifically includes:

[0032] screen out valid matching pixel points from each level of concentric ring search region to form a set of valid matching pixel points at each level;

[0033] calculate the parallax median of the set of valid matching pixel points at each level to obtain a parallax median at each level;

[0034] fit a distribution model of parallax change with radius according to the parallax medians at each level to obtain a parallax distribution model;

[0035] Calculate a predicted disparity value of the matching abnormal pixel point based on the disparity distribution model, and use the predicted disparity value as a disparity correction parameter of the matching abnormal pixel point in matching of the remote sensing image.

[0036] In some embodiments, the disparity correction parameter is used to correct the disparity of the matching abnormal pixel point, and all corrected disparity points of the target forest region are obtained.

[0037] Extract the original disparity value corresponding to the matching abnormal pixel point and the disparity correction parameter;

[0038] Calculate the deviation of the original disparity value and the disparity correction parameter;

[0039] Correct the original disparity value of the matching abnormal pixel point based on the deviation to obtain a corrected disparity value;

[0040] Assign the corrected disparity value to the corresponding matching abnormal pixel point, and obtain all corrected disparity points of the target forest region.

[0041] In some embodiments, the pair of forest remote sensing images comprises two remote sensing images with disparity information of the same forest scene.

[0042] In some embodiments, the pair of forest remote sensing images of the target forest region are obtained by synchronously imaging the target forest region by a high-resolution camera.

[0043] In a second aspect, the present application provides a high-resolution remote sensing image forest three-dimensional reconstruction system, comprising:

[0044] An acquisition module is configured to acquire a pair of forest remote sensing images of a target forest region, perform pixel-level disparity analysis on the pair of forest remote sensing images, and generate a forest disparity map of the target forest region.

[0045] A processing module is configured to locate matching abnormal pixel points in matching of the remote sensing image from the pair of forest remote sensing images based on a disparity mapping relationship of the pixel points in the forest disparity map.

[0046] The processing module is further configured to perform patch segmentation on the pair of forest remote sensing images of the target forest region based on the edge contour continuity of the forest canopy, and obtain independent closed patches with tree crown topological structure as a boundary.

[0047] The processing module is further configured to, for the independent closed patch in which the matching abnormal pixel point is located, construct a multi-level concentric ring search area with the matching abnormal pixel point as the center in the independent closed patch, wherein the maximum search radius of the multi-level concentric ring search area is constrained by the minimum circumscribed circle of the independent closed patch.

[0048] The execution module is configured to determine a parallax correction parameter of a matching abnormal pixel point in remote sensing image matching according to valid matching pixel points in a search area of each concentric circle ring, to perform parallax correction on the matching abnormal pixel point according to the parallax correction parameter, and to obtain all corrected parallax points of the target forest region, and to reconstruct a forest three-dimensional point cloud model of the target forest region based on all the corrected parallax points

[0049] The technical scheme provided by the embodiments disclosed in the application has the following beneficial effects:

[0050] In the high-resolution remote sensing image forest three-dimensional reconstruction method and system provided by the application, first.

[0051] It can be seen that the application can overcome the destructive influence of matching abnormal points caused by the complex structure of the forest canopy on the three-dimensional reconstruction accuracy. First, the forest parallax map is generated by performing pixel-level parallax analysis on the forest remote sensing image pair, which provides basic parallax data for subsequent abnormal point positioning and three-dimensional reconstruction. Second, the matching abnormal pixel points are located using the parallax mapping relationship, which can accurately identify the error matching area and avoid the interference of the error matching area on the reconstruction accuracy. Further, the independent closed patches are obtained based on the continuity of the canopy edge contour, which can effectively fit the forest boundary in the target forest region and provide an accurate spatial range for targeted processing of matching abnormal pixel points. Then, the multi-level concentric circle search area constrained by the minimum circumscribed circle is constructed in the independent closed patch, which can efficiently extract local valid matching information and improve the reliability of the correction parameter. Finally, the parallax correction parameter is determined in combination with the valid matching pixel points and the correction is completed, and the forest three-dimensional point cloud model reconstructed based on the corrected parallax points can obtain a forest three-dimensional state with higher detail fidelity and spatial accuracy in a high-density and complex-structure forest scene, which provides high-quality data support for forest parameter extraction and growth monitoring, and further avoids the non-continuous parallax jump caused by image matching due to the mutual occlusion of forest canopy leaves and high-frequency texture characteristics. In summary, the technical scheme provided by the application can overcome the destructive influence of matching abnormal points caused by the complex structure of the forest canopy on the three-dimensional reconstruction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is an application scenario architecture schematic diagram of the high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the application;

[0053] Figure 2 is an exemplary flowchart of the high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the application;

[0054] Figure 3 is an exemplary flowchart of determining a matching abnormal pixel point according to some embodiments of the application;

[0055] Figure 4 is a structural schematic diagram of a high-resolution remote sensing image forest three-dimensional reconstruction system according to some embodiments of the present application;

[0056] Figure 5 is a structural schematic diagram of a computer device for implementing a high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the present application. DETAILED DESCRIPTION

[0057] In order to better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in combination with the drawings in the specification and specific embodiments.

[0058] Reference Figure 1 The figure is a schematic diagram of an application scenario architecture of a high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the present application. The application scenario architecture includes a collection terminal, a communication network and a server end. The collection terminal and the server end are directly or indirectly connected through the communication network. The collection terminal acquires a pair of forest remote sensing images of a target forest area and uploads them to the server terminal. The server end performs pixel-level disparity analysis on the pair of forest remote sensing images to generate a forest disparity map of the target forest area. The matching abnormal pixel points in the remote sensing image matching are located from the pair of forest remote sensing images through the disparity mapping relationship of the pixel points in the forest disparity map. Based on the edge contour continuity of the forest canopy, the forest remote sensing images of the target forest area are segmented into independent closed patches with tree crown topological structure as the boundary. For the independent closed patch where the matching abnormal pixel points are located, a multi-level concentric ring search area is constructed with the matching abnormal pixel points as the center in the independent closed patch. The maximum search radius of the multi-level concentric ring search area is constrained by the minimum circumscribed circle of the independent closed patch. The disparity correction parameters of the matching abnormal pixel points in the remote sensing image matching are determined according to the valid matching pixel points in each level of the concentric ring search area. The disparity correction parameters are used to correct the disparity of the matching abnormal pixel points, and then all the corrected disparity points of the target forest area are obtained. Based on all the corrected disparity points, a forest three-dimensional point cloud model of the target forest area is reconstructed.

[0059] Reference Figure 2 The figure is an exemplary flowchart of a high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the present application. The high-resolution remote sensing image forest three-dimensional reconstruction method mainly includes the following steps:

[0060] In step 101, a pair of forest remote sensing images of a target forest area is acquired. Pixel-level disparity analysis is performed on the pair of forest remote sensing images to generate a forest disparity map of the target forest area.

[0061] In a specific implementation, the target forest area is synchronously imaged by a high-resolution camera to obtain a pair of forest remote sensing images of the target forest area, the pair of forest remote sensing images being two remote sensing images containing the same forest scene and having parallax information.

[0062] It should be noted that the pair of forest remote sensing images of the same target forest area obtained accurately can provide a geometric reference for subsequent pixel-level parallax analysis, same-name point matching and three-dimensional coordinate inversion by using the parallax information formed due to the difference in observation angles between the images.

[0063] In some embodiments, the pixel-level parallax analysis of the pair of forest remote sensing images to generate the forest parallax map of the target forest area can be implemented by the following steps, i.e.:

[0064] The pair of forest remote sensing images is geometrically corrected to obtain a pair of corrected forest remote sensing images;

[0065] Same-name feature point pairs are extracted from the pair of corrected forest remote sensing images to form a set of same-name feature point pairs;

[0066] The parallax values of corresponding pixel points are calculated for each same-name feature point pair in the set of same-name feature point pairs to obtain all parallax values;

[0067] The forest parallax map of the target forest area is constructed according to all parallax values.

[0068] In a specific implementation, first, a cubic polynomial model is used to fit the geometric distortion of the forest remote sensing image pair, and then a cubic convolution interpolation method is used to resample the image pixels in the forest remote sensing image pair, and a corrected forest remote sensing image pair is obtained, wherein the corrected forest remote sensing image pair represents two remote sensing images in which the geometric deformation is eliminated; second, the same-named feature points are extracted from the corrected forest remote sensing image pair to form a same-named feature point set, that is, a Gaussian pyramid is constructed for each corrected forest remote sensing image in the corrected forest remote sensing image pair to generate a multi-scale space corresponding to each corrected forest remote sensing image, and local extreme points are detected as feature points in different scale spaces, and then a 128-dimensional feature vector (i.e., a feature descriptor) containing gradient information is established based on the neighborhood gradient direction of the feature points, and then the Euclidean distance of the feature vectors between the feature points in the corrected forest remote sensing image pair is calculated, and the feature point pairs with a Euclidean distance of the feature vectors less than a set threshold are taken as the same-named feature point pairs to form a same-named feature point pair set, wherein the same-named feature point pair set refers to a set of pixel points corresponding to the same position in the two corrected forest remote sensing images in the corrected forest remote sensing image pair; then, a forest disparity map of the target forest region is constructed according to all the disparity values, that is, the disparity value of each pixel point in each same-named feature point pair in the same-named feature point pair set is calculated to obtain all the disparity values, and the disparity value is the pixel difference of the two pixel points corresponding to the same-named feature point pair; finally, a forest disparity map of the target forest region is constructed according to all the disparity values, that is, all the pixel disparity values are arranged according to their spatial positions in the remote sensing image to form a two-dimensional matrix reflecting the disparity distribution, and then the forest disparity map of the target forest region is constructed.

[0069] It should be noted that the forest disparity map of the target forest region in the present application represents a set of pixel point disparity information in the target forest region in the form of an image, wherein the disparity value represents the horizontal offset of the imaging position of the same position point in the forest remote sensing image pair, and in the three-dimensional reconstruction of the forest image, the accurate determination of the forest disparity map can provide reliable basic data support for the subsequent construction of the three-dimensional point cloud model, effectively reduce the disparity error caused by the complex situations such as the shelter of the branches and leaves of the forest canopy, the similarity of the texture, etc., and improve the accuracy of the disparity information, thereby providing a more reasonable disparity basis for the subsequent triangular measurement of the three-dimensional coordinates, and further improving the detail performance and overall reconstruction accuracy of the forest three-dimensional point cloud model.

[0070] In step 102, the matching abnormal pixel points in the remote sensing image matching are located from the forest remote sensing image pair through the disparity mapping relationship of the pixel points in the forest disparity map.

[0071] In some embodiments, with reference to Figure 3As shown, the figure is an exemplary flow chart for determining matching abnormal pixel points according to some embodiments of the present application. In this embodiment, the matching abnormal pixel points in the remote sensing image matching process can be determined by the parallax mapping relationship of the pixels in the forest parallax map.

[0072] First, in step 1021, the parallax threshold range in the remote sensing image matching process is determined based on the parallax mapping relationship of the pixels in the forest parallax map.

[0073] Then, in step 1022, the pixels with parallax values exceeding the parallax threshold range are screened out from the forest parallax map as candidate abnormal points.

[0074] Finally, in step 1023, the pixels corresponding to the candidate abnormal points in the forest remote sensing image pair are taken as the matching abnormal pixel points in the remote sensing image matching process.

[0075] In specific implementation, first, the parallax mapping relationship of the pixels in the forest parallax map is obtained, which is characterized by pixel parallax values. The mean and standard deviation of all pixel parallax values in the forest parallax map are calculated by statistical analysis method, and then the upper and lower limits of the threshold are set according to the normal distribution principle (such as mean plus or minus 3 times standard deviation), so as to obtain the parallax threshold range in the remote sensing image matching process. The parallax threshold range represents the numerical limit for determining whether the pixel parallax value is in a reasonable range. Then, the pixels with parallax values exceeding the parallax threshold range are screened out from the forest parallax map as candidate abnormal points, which are the pixels preliminarily determined to have matching errors. Then, the pixels corresponding to the candidate abnormal points in the forest remote sensing image pair are taken as the matching abnormal pixel points in the remote sensing image matching process.

[0076] It should be noted that the matching abnormal pixel points in the present application represent the pixel points with matching errors in the remote sensing image matching process. Through the determination of the matching abnormal pixel points, the error matching area caused by factors such as branch and leaf shielding, texture repetition and illumination change in the forest remote sensing image matching process can be accurately identified, which provides a clear target for subsequent targeted parallax correction, avoids the interference of abnormal values on the accuracy of the overall parallax map, and at the same time, this process can reduce the influence of invalid data on three-dimensional reconstruction, ensuring that the final generated forest point cloud model is more consistent with the actual scene characteristics in terms of detail integrity and spatial position accuracy, and providing a reliable data basis for subsequent forest parameter extraction, growth state monitoring and other applications.

[0077] In step 103, the forest remote sensing image of the target forest region is segmented based on the edge contour continuity of the forest canopy, to obtain independent closed patches with tree crown topological structure as the boundary.

[0078] In some embodiments, patch segmentation of a forest remote sensing image of a target forest region based on continuity of edge contours of forest canopies can be achieved by the following steps, i.e.:

[0079] Obtaining a forest remote sensing image of a target forest region, and performing Gaussian filtering denoising on the forest remote sensing image to obtain a smoothed forest remote sensing image;

[0080] Extracting a canopy edge contour map from the smoothed forest remote sensing image;

[0081] Performing a morphological connection operation on the canopy edge contour map to obtain a continuous canopy edge contour of a forest canopy;

[0082] Based on the continuous canopy edge contour, performing initial segmentation on the forest remote sensing image using a region growing algorithm to obtain initial patches;

[0083] According to the characteristics of the tree crown topological structure, performing morphological correction on the initial patches to obtain independent closed patches with the tree crown topological structure as the boundary.

[0084] In a specific implementation, first, a high-resolution camera is used to obtain a forest remote sensing image of a target forest region, and a Gaussian filter is used to denoise the forest remote sensing image to obtain a smoothed forest remote sensing image, where the smoothed forest remote sensing image is a remote sensing image in which main feature information is retained and noise is suppressed. Next, a Canny edge detection algorithm can be used to extract a crown layer edge profile map from the smoothed forest remote sensing image, where the crown layer edge profile map represents a set of edge lines reflecting the shape of the forest crown layer boundary. Further, a morphological connection operation is performed on the crown layer edge profile map to obtain a continuous crown layer edge profile of the forest crown layer. Specifically, a morphological dilation operation (using a suitable structural element, such as a 3x3 square structure) is used to expand the edge profile in the crown layer edge profile map, so that broken edge segments are connected, and then an erosion operation is used to restore the original width of the edge to obtain the continuous crown layer edge profile, where the continuous crown layer edge profile is a crown layer profile in which the edge lines are complete and have no obvious breaks after morphological processing. Then, based on the continuous crown layer edge profile, a region growing algorithm is used to perform initial segmentation on the forest remote sensing image to obtain an initial patch. Specifically, the region growing algorithm uses the pixels within the crown layer edge in the continuous crown layer edge profile as seed points, sets a gray value similarity threshold, and merges pixels that are adjacent to the seed points and have a gray value difference within the gray value similarity threshold into the same region to generate the initial patch. The gray value similarity threshold can be set according to actual requirements or according to expert knowledge, and is not limited herein. The initial patch is a preliminary segmentation region bounded by the continuous crown layer edge. Finally, morphological correction is performed on the initial patch according to the topological structure characteristics of the tree crown to obtain an independent and closed patch bounded by the topological structure of the tree crown. Specifically, an opening operation is used to remove small protrusions on the edge of the initial patch based on the fact that the tree crown usually has a circular or elliptical topological structure, and then a closing operation is used to fill small holes in the initial patch to obtain the independent and closed patch bounded by the topological structure of the tree crown.

[0085] It should be noted that the independent and closed patch in this application represents a complete and closed region of the target forest region that conforms to the natural growth pattern of the tree crown. In this embodiment, the continuity of the forest crown layer edge profile is used as the basis for morphological operations and correction of the topological characteristics of the tree crown, so that the segmented patch can accurately fit the spatial boundary of a single tree, effectively avoiding the problems of patch adhesion or boundary ambiguity caused by dense growth of trees in traditional methods, improving the accuracy of single tree segmentation, and uniquely using the topological structure of the tree crown as the core constraint for patch segmentation. Through the cooperative processing of Gaussian filter denoising, Canny edge detection, morphological connection, and region growing, an independent and closed unit is constructed that retains the details of the crown layer and conforms to the growth pattern of the forest, providing a precise spatial range for subsequent correction of matching abnormal pixel points for a single tree, and laying a foundation for fine processing of three-dimensional reconstruction.

[0086] In step 104, for the independent closed patch where the matched abnormal pixel point is located, a multi-level concentric ring search region is constructed in the independent closed patch with the matched abnormal pixel point as the center, wherein the maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumscribed circle of the independent closed patch.

[0087] In some embodiments, for the independent closed patch where the matched abnormal pixel point is located, the multi-level concentric ring search region can be constructed in the independent closed patch with the matched abnormal pixel point as the center by the following steps, that is:

[0088] determining the minimum circumscribed circle radius of the independent closed patch where the matched abnormal pixel point is located;

[0089] taking the minimum circumscribed circle radius as the maximum search radius of the multi-level concentric ring search region, and setting the radius increment step of the concentric ring;

[0090] starting from the initial radius, constructing each level of the concentric ring in turn according to the radius increment step with the matched abnormal pixel point as the center, until the maximum search radius is reached;

[0091] cutting each level of the concentric ring according to the boundary of the independent closed patch to form the multi-level concentric ring search region in the independent closed patch.

[0092] In a specific implementation, first, the minimum circumscribed circle radius of the independent closed patch where the matching abnormal pixel point is located is determined, that is, the rotation puzzle algorithm is used to traverse all pixel points on the boundary of the independent closed patch, the radius of the circle that can completely contain the independent closed patch and has the minimum radius is calculated, and the minimum circumscribed circle radius of the independent closed patch where the matching abnormal pixel point is located is obtained. The minimum circumscribed circle radius refers to the radius of the smallest circle that can surround the independent closed patch. Second, the minimum circumscribed circle radius is used as the maximum search radius of the multi-level concentric ring search area, and the radius increment step of the concentric ring is set. The radius increment step can be set according to the distribution density of the pixel points in the patch or set according to actual needs, which is not limited here. The radius increment step refers to the radius difference between adjacent two levels of concentric rings. Then, starting from the initial radius, each level of concentric ring is constructed in turn according to the radius increment step, until the maximum search radius is reached, that is, the coordinates of the matching abnormal pixel point are taken as the center, starting from the set initial radius (which can be set as the minimum step unit), a circle is drawn each time by increasing a radius increment step, until the circle radius reaches the maximum search radius, and then each level of concentric ring is constructed. Each level of concentric ring refers to a plurality of circular ring bands centered on the matching abnormal pixel point and having radii that increase in turn by the increment step. Finally, the boundary of the independent closed patch is used to crop each level of concentric ring to form a multi-level concentric ring search area in the independent closed patch, that is, the ray method is used to determine whether the pixel points on each level of concentric ring are located inside the independent closed patch, the part located inside the independent closed patch is retained, and the part outside the boundary of the independent closed patch is removed, and then a multi-level concentric ring search area in the independent closed patch is formed.

[0093] It should be noted that the multi-level concentric ring search area in this application represents the search area formed by each level of concentric ring located inside the independent closed patch. In this embodiment, the minimum circumscribed circle of the independent closed patch is used to constrain the maximum search radius, and the hierarchical ring band is constructed in combination with the radius increment step. This not only avoids the irrelevant pixel interference introduced by the too large search range, but also realizes the hierarchical extraction of local effective information through step-by-step expansion, improves the pertinence of abnormal point correction, and creatively uses the patch boundary as the natural constraint of the search area. Through the ray method, the search range is strictly limited within the single tree crown, and the multi-level ring band structure is used to capture the parallax distribution law of the neighborhood at different distances, so that the calculation of the subsequent parallax correction parameter is more in line with the local morphological characteristics of the tree crown. Especially in a complex crown scene, the search accuracy and calculation efficiency can be effectively balanced, and structured neighborhood information support is provided for accurate correction of matching abnormal pixel points.

[0094] In step 105, the parallax correction parameters of the matching abnormal pixel points in the remote sensing image matching are determined according to the effective matching pixel points in the search area of each concentric circle ring, the parallax correction parameters are used for parallax correction of the matching abnormal pixel points, and then all the corrected parallax points of the target forest region are obtained. The three-dimensional point cloud model of the forest region of the target forest region is reconstructed based on all the corrected parallax points.

[0095] In some embodiments, the parallax correction parameters of the matching abnormal pixel points in the remote sensing image matching can be determined according to the effective matching pixel points in the search area of each concentric circle ring, and the following steps can be used to achieve the determination, that is:

[0096] The effective matching pixel points are screened out from the search area of each concentric circle ring to form a set of effective matching pixel points of each level;

[0097] The parallax median of the set of effective matching pixel points of each level is calculated to obtain the parallax median of each level;

[0098] A distribution model of the parallax changing with the radius is fitted according to the parallax medians of each level to obtain the parallax distribution model;

[0099] The predicted parallax value of the matching abnormal pixel points is calculated based on the parallax distribution model, and the predicted parallax value is used as the parallax correction parameter of the matching abnormal pixel points in the remote sensing image matching.

[0100] In a specific implementation, first, valid matching pixel points are screened from each level of the concentric circular ring search area to form a set of valid matching pixel points for each level, that is, for each level of the concentric circular ring search area, the standard deviation and mean value of all disparity values in the concentric circular ring search area are calculated, the pixel points with disparity values within the range of the mean value plus or minus 3 times the standard deviation are regarded as valid matching pixel points, and all the valid matching pixel points are combined into a set of valid matching pixel points, and then valid matching pixel points are screened from each level of the concentric circular ring search area to form a set of valid matching pixel points for each level, which refers to a set of pixel points with stable disparity in each level of the concentric circular ring search area; second, the disparity values of the pixel points in each set of valid matching pixel points are sorted in descending order, and the disparity value at the middle position is taken as the median disparity of the corresponding level to obtain the median disparity of each level, which refers to the disparity value at the middle position after the disparity values of each set of valid matching pixel points are sorted in descending order, and is used to measure the central tendency of the disparity values in the set of valid matching pixel points; then, a distribution model of the change of disparity with radius is fitted according to the median disparity of each level to obtain the disparity distribution model, that is, the radius of each level of the concentric circular ring is taken as the independent variable, and the corresponding median disparity is taken as the dependent variable, and a polynomial fitting (such as a quadratic polynomial) is performed by using the least square method to obtain a function model that can reflect the change law of the disparity with the radius, that is, the disparity distribution model, which refers to a function model that can reflect the change law of the disparity with the radius of the concentric circular ring; finally, the predicted disparity value of the matching abnormal pixel point is calculated based on the disparity distribution model, and the predicted disparity value is taken as the disparity correction parameter of the matching abnormal pixel point in the matching of the remote sensing image, that is, the distance between the matching abnormal pixel point and the nearest matching normal pixel point is taken as the output radius, which is substituted into the disparity distribution model to obtain the function value as the predicted disparity value of the matching abnormal pixel point, and the predicted disparity value is taken as the disparity correction parameter of the matching abnormal pixel point in the matching of the remote sensing image, which refers to the predicted disparity value used to correct the disparity of the matching abnormal pixel point.

[0101] It should be noted that the disparity correction parameter in this application represents the parameter value used to correct the disparity of the matching abnormal pixel point, and in this embodiment, valid matching pixel points are screened by multi-level concentric circular ring layering, and the robust statistical characteristics of the median disparity are combined to reduce the interference of local noise on parameter estimation, and a polynomial fitting is used to construct the distribution model of the change of disparity with radius, so that the correction parameter can accurately reflect the local disparity change law of the forest canopy, and the accuracy of the abnormal point correction is improved. The spatial hierarchical structure of the concentric circular ring is combined with the radius distribution characteristics of the disparity, and the model fitting based on the median of each level retains the gradient change of the neighborhood information, so that the correction parameter is more suitable for the three-dimensional morphological characteristics of the forest, and provides key support for the detail fidelity of subsequent point cloud reconstruction.

[0102] In some embodiments, the disparity correction parameter is used to correct the matching abnormal pixel points, and all corrected disparity points of the target forest region are obtained by the following steps:

[0103] The original disparity value and the disparity correction parameter corresponding to the matching abnormal pixel points are extracted.

[0104] The deviation between the original disparity value and the disparity correction parameter is calculated.

[0105] The original disparity value of the matching abnormal pixel points is corrected based on the deviation to obtain a corrected disparity value.

[0106] The corrected disparity value is assigned to the corresponding matching abnormal pixel points, and all corrected disparity points of the target forest region are obtained.

[0107] In the specific implementation, first, the matching abnormal pixel points marked in the forest disparity map are traversed, and the corresponding original disparity value and disparity correction parameter in the forest disparity map are read. The original disparity value refers to the disparity value of the matching abnormal pixel points before correction. Second, when calculating the deviation between the original disparity value and the disparity correction parameter, the original disparity value is subtracted from the disparity correction parameter by using subtraction operation to obtain the deviation between the original disparity value and the disparity correction parameter. The deviation refers to the difference between the original disparity value and the disparity correction parameter. Third, when correcting the original disparity value of the matching abnormal pixel points based on the deviation to obtain a corrected disparity value, the corrected disparity value is obtained by subtracting the deviation from the original disparity value. The corrected disparity value refers to the disparity value of the matching abnormal pixel points after correction by the disparity correction parameter. Finally, the corrected disparity value is assigned to the corresponding matching abnormal pixel points, and all corrected disparity points of the target forest region are obtained. That is, the original disparity value of the corresponding matching abnormal pixel points is replaced by the calculated corrected disparity value, while the disparity values of other normally matched pixel points in the forest disparity map are retained, and all corrected disparity points of the target forest region are obtained.

[0108] It should be noted that all the corrected disparity points in the present application refer to a set of corrected matching abnormal pixel points and normally matched pixel points in the target forest region. Through the determination of the corrected disparity points, the disparity error of the matching abnormal pixel points can be eliminated, the disparity information is more consistent with the actual spatial distribution characteristics of the forest, and accurate basic data are provided for three-dimensional point cloud reconstruction. At the same time, this process integrates the disparity information of the normally matched pixel points and the corrected pixel points, ensures the completeness and reliability of the disparity data of the target forest region, and further improves the accuracy of subsequent triangular measurement of three-dimensional coordinates.

[0109] In some embodiments, the forest three-dimensional point cloud model of the target forest region is reconstructed based on all the corrected disparity points by the following steps:

[0110] obtaining the internal parameters and the external parameters of the pair of forest remote sensing images;

[0111] calculating the three-dimensional coordinates of each corrected parallax point according to the internal parameters, the external parameters and all the corrected parallax points through the principle of triangulation;

[0112] performing coordinate conversion on the three-dimensional coordinates of all the corrected parallax points to unify them to a preset coordinate system to form a three-dimensional point cloud;

[0113] constructing a forest three-dimensional point cloud model of a target forest region based on the three-dimensional point cloud.

[0114] In the specific implementation, firstly, when the internal parameters and the external parameters of the pair of forest remote sensing images are obtained, the internal parameters (including focal length, principal point coordinates and distortion coefficients) of the camera are calculated through a camera calibration method (such as Zhang Zhengyou calibration method) by using the captured checkerboard calibration board image, and then the rotation matrix and the translation vector of the pair of forest remote sensing images are solved as the external parameters through a relative orientation method, wherein the internal parameters refer to the parameters describing the optical characteristics and imaging geometric relationship of the camera itself, and the external parameters refer to the parameters describing the relative position and attitude of the two images in space; secondly, the three-dimensional coordinates of each corrected parallax point are calculated according to the internal parameters, the external parameters and all the corrected parallax points through the principle of triangulation, that is, the pixel coordinates of the corrected parallax points in the pair of forest remote sensing images are respectively substituted into the camera imaging model, the space geometric relationship is constructed in combination with the internal parameters and the external parameters, and the three-dimensional coordinates of the corrected parallax points are solved according to the parallax information of the corresponding points in the pair of forest remote sensing images, wherein the three-dimensional coordinates refer to the corresponding position coordinates of each corrected parallax point in the three-dimensional space; then, the three-dimensional coordinates of each corrected parallax point are converted from the image coordinate system to a preset coordinate system (i.e. the geodetic coordinate system) through a coordinate transformation matrix (solved through an absolute orientation method) to form a three-dimensional point cloud, wherein the three-dimensional point cloud refers to a point set composed of three-dimensional coordinate points and reflecting the spatial form of the target forest region; finally, the forest three-dimensional point cloud model of the target forest region is constructed based on the three-dimensional point cloud, that is, the three-dimensional point cloud is denoised (such as by using statistical filtering to remove outliers), and the coordinate points reflecting the characteristics of the forest canopy and branches are retained, and then the forest three-dimensional point cloud model of the target forest region is obtained.

[0115] It should be noted that the forest three-dimensional point cloud model of the target forest region in the present application refers to a three-dimensional point cloud set that can completely present the spatial structure and form characteristics of the forest in the target forest region, and through the determination of the forest three-dimensional point cloud model, the two-dimensional remote sensing image information can be converted into three-dimensional spatial structure data, and the canopy form, branch distribution and spatial position relationship of the single forest in the target forest region can be intuitively and accurately presented, thereby providing quantifiable three-dimensional parameters for forest resource investigation.

[0116] In addition, another aspect of the present application provides, in some embodiments, a high-resolution remote sensing image forest three-dimensional reconstruction system, which refers to Figure 4 FIG. 2 is a structural schematic diagram of a high-resolution remote sensing image forest three-dimensional reconstruction system according to some embodiments of the present application, which includes an acquisition module 201, a processing module 202, and an execution module 203, which are described as follows:

[0117] The acquisition module 201 is mainly used for acquiring a pair of remote sensing images of a target forest area in the present application, performing pixel-level disparity analysis on the pair of remote sensing images of the forest, and generating a forest disparity map of the target forest area.

[0118] The processing module 202 is mainly used for locating matching abnormal pixels in remote sensing image matching from the pair of remote sensing images of the forest through the disparity mapping relationship of the pixels in the forest disparity map in the present application.

[0119] The processing module 202 is also used for performing patch segmentation on the remote sensing images of the target forest area based on the edge contour continuity of the forest canopy, so as to obtain independent closed patches with tree crown topological structure as the boundary.

[0120] In addition, the processing module 202 is also used for constructing a multi-level concentric ring search area with the matching abnormal pixels as the center in the independent closed patch where the matching abnormal pixels are located, wherein the maximum search radius of the multi-level concentric ring search area is constrained by the minimum circumscribed circle of the independent closed patch.

[0121] The execution module 203 is mainly used for determining the disparity correction parameters of the matching abnormal pixels in remote sensing image matching according to the valid matching pixels in each level of the concentric ring search area, performing disparity correction on the matching abnormal pixels according to the disparity correction parameters, and then obtaining all the corrected disparity points of the target forest area, and reconstructing the forest three-dimensional point cloud model of the target forest area based on all the corrected disparity points.

[0122] In addition, the present application also provides a computer device, which includes a memory and a processor, the memory stores codes, and the processor is configured to acquire the codes and execute the high-resolution remote sensing image forest three-dimensional reconstruction method described above.

[0123] In some embodiments, referring to Figure 5 FIG. 2 is a structural schematic diagram of a computer device for implementing the high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the present application. The high-resolution remote sensing image forest three-dimensional reconstruction method in the above embodiments can be implemented by the computer device shown in FIG. 2. Figure 5The computer device shown is implemented by a computer device including at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0124] The processor 301 can be a general central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of the high-resolution remote sensing image forest three-dimensional reconstruction method in the application.

[0125] The communication bus 302 can be used to transmit information between the above-mentioned components.

[0126] The memory 303 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, an optical disk storage (including a compact disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited to this. The memory 303 can exist independently and be connected to the processor 301 through the communication bus 302. The memory 303 can also be integrated with the processor 301.

[0127] The memory 303 is used to store program codes for executing the scheme of the application, and the processor 301 is used to control the execution. The processor 301 is used to execute the program codes stored in the memory 303. The program codes can include one or more software modules. The determination of the high-resolution remote sensing image forest three-dimensional reconstruction method in the above-mentioned embodiments can be realized by the processor 301 and one or more software modules in the program codes in the memory 303.

[0128] The communication interface 304, using any transceiver-like mechanism, is used to communicate with other devices or communication networks, such as an Ethernet network, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0129] In a particular implementation, as one example, a computer device can include multiple processors, each of which can be a single-CPU processor or a multi-CPU processor. A processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0130] The computer device described above can be a general-purpose computer device or a special-purpose computer device. In a particular implementation, the computer device can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0131] In addition, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the high-resolution remote sensing image forest three-dimensional reconstruction method.

[0132] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to cover all changes and modifications falling within the scope of the present application.

[0133] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A method for three-dimensional reconstruction of forest trees from high-resolution remote sensing images, characterized in that, Includes the following steps: Acquire a pair of remote sensing images of trees in the target forest area, perform pixel-level disparity analysis on the pair of remote sensing images of trees, and generate a disparity map of trees in the target forest area; By using the disparity mapping relationship of pixels in the tree disparity map, the abnormal matching pixels during remote sensing image matching are located from the tree remote sensing image pair. Based on the continuity of the edge contour of the tree canopy, the remote sensing image of the target forest area is segmented into patches to obtain independent closed patches with the tree canopy topology as the boundary. For an independent closed patch containing a matching abnormal pixel, a multi-level concentric ring search region is constructed within the independent closed patch with the matching abnormal pixel as the center. The maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumcircle of the independent closed patch. Based on the effective matching pixels within the search area of ​​each concentric ring, the disparity correction parameters for abnormal matching pixels during remote sensing image matching are determined. The disparity correction parameters are used to correct the disparity of the abnormal matching pixels, thereby obtaining all corrected disparity points of the target forest area. Based on all corrected disparity points, the three-dimensional point cloud model of the target forest area is reconstructed.

2. The method as described in claim 1, characterized in that, Pixel-level disparity analysis is performed on the aforementioned forest remote sensing image pairs to generate a forest disparity map of the target forest area, specifically including: Geometric correction is performed on the tree remote sensing image pairs to obtain corrected tree remote sensing image pairs; Extract pairs of feature points with the same name from the corrected forest remote sensing image pairs to form a set of feature point pairs with the same name; For each pair of feature points in the set of identical feature points, calculate the disparity value of the corresponding pixel to obtain all disparity values; Construct a tree disparity map of the target forest area based on all disparity values.

3. The method as described in claim 1, characterized in that, Specifically, the abnormal matching pixels in the remote sensing image pair are located from the tree remote sensing image pair by using the disparity mapping relationship of the pixels in the tree disparity map. These include: The disparity threshold range in the remote sensing image matching process is determined based on the disparity mapping relationship of pixels in the tree disparity map. Pixels whose disparity values ​​exceed the disparity threshold range are selected from the tree disparity map as candidate anomalies; The pixels corresponding to the candidate anomalies in the forest remote sensing images are used as the matching anomaly pixels when matching remote sensing images.

4. The method as described in claim 1, characterized in that, Based on the continuity of the tree canopy edge contour, patch segmentation is performed on the remote sensing image of the target forest area to obtain independent closed patches with the tree canopy topology as the boundary. Specifically, these patches include: Acquire remote sensing images of trees in the target forest area, and perform Gaussian filtering to denoise the tree remote sensing images to obtain smooth tree remote sensing images; Extract the canopy edge contour map from the smooth forest remote sensing image; A morphological connection operation is performed on the canopy edge contour map to obtain a continuous canopy edge contour of the forest canopy; Based on the continuous canopy edge contour, the forest remote sensing image is initially segmented using a region growing algorithm to obtain initial patches; The initial patches are morphologically corrected based on the topological characteristics of the tree canopy to obtain independent closed patches with the tree canopy topology as the boundary.

5. The method as described in claim 1, characterized in that, For the isolated closed patch containing the anomalous matching pixel, a multi-level concentric ring search region is constructed within the isolated closed patch centered on the anomalous matching pixel. Specifically, this includes: Determine the minimum circumcircle radius of the isolated closed patch containing the matched abnormal pixel; The minimum circumscribed circle radius is used as the maximum search radius of the multi-level concentric ring search area, and the radius increment step of the concentric rings is set. Using the matched abnormal pixel as the center, starting from the initial radius, concentric rings of each level are constructed sequentially according to the radius increment step size until the maximum search radius is reached; Based on the boundaries of the independent closed patches, concentric rings at all levels are trimmed to form a multi-level concentric ring search area within the independent closed patches.

6. The method as described in claim 1, characterized in that, The disparity correction parameters for mismatched pixels during remote sensing image matching are determined based on the effective matching pixels within the search area of ​​each concentric ring. These parameters include: Valid matching pixels are selected from the search areas of each concentric ring to form a set of valid matching pixels for each level. Calculate the median disparity of the effective matching pixel set at each level to obtain the median disparity at each level; The disparity distribution model is obtained by fitting a disparity distribution model as a function of radius to the median of disparity at each level; The predicted disparity value for matching abnormal pixels is calculated based on the disparity distribution model, and the predicted disparity value is used as the disparity correction parameter for matching abnormal pixels when matching remote sensing images.

7. The method as described in claim 1, characterized in that, Based on the disparity correction parameters, disparity correction is performed on the matched abnormal pixels to obtain all corrected disparity points of the target forest area, specifically including: Extract the original disparity value and disparity correction parameters corresponding to the matched abnormal pixels; Calculate the deviation between the original disparity value and the disparity correction parameter; The original disparity value of the matched abnormal pixel is corrected based on the deviation to obtain the corrected disparity value; The corrected disparity value is assigned to the corresponding matched abnormal pixel, thereby obtaining all corrected disparity points of the target forest area.

8. The method as described in claim 1, characterized in that, The forest remote sensing image pair consists of two remote sensing images containing the same forest scene and having parallax information.

9. The method as described in claim 1, characterized in that, The target forest area is simultaneously imaged using a high-resolution camera to obtain remote sensing image pairs of trees in the target forest area.

10. A high-resolution remote sensing image forest three-dimensional reconstruction system, characterized in that, The system includes: The acquisition module is used to acquire remote sensing image pairs of trees in the target forest area, perform pixel-level disparity analysis on the tree remote sensing image pairs, and generate a tree disparity map of the target forest area. The processing module is used to locate the abnormal matching pixels in the remote sensing image pair from the tree remote sensing image pair by using the disparity mapping relationship of the pixels in the tree disparity map; The processing module is also used to perform patch segmentation on the remote sensing image of the target forest area based on the continuity of the edge contour of the tree canopy, so as to obtain independent closed patches with the tree canopy topology as the boundary; The processing module is also used to construct a multi-level concentric ring search region centered on the matching abnormal pixel within the independent closed patch where the matching abnormal pixel is located. The maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumcircle of the independent closed patch. The execution module is used to determine the disparity correction parameters of the abnormal matching pixels when matching remote sensing images based on the effective matching pixels within the search area of ​​each concentric ring. Based on the disparity correction parameters, the abnormal matching pixels are disparity corrected to obtain all corrected disparity points of the target forest area. Based on all corrected disparity points, the three-dimensional point cloud model of the target forest area is reconstructed.

Images