High-resolution remote sensing image forest tree three-dimensional reconstruction method and system

Abstract

The invention provides a high-resolution remote sensing image forest three-dimensional reconstruction method and system. The method comprises the following steps: generating a forest disparity map of a target forest region through a forest remote sensing image pair of a target forest; extracting abnormal matching pixel points during remote sensing image matching through a parallax mapping relation of the pixel points in the forest disparity map; extracting an independent closed plaque taking the crown topological structure as a boundary from the forest remote sensing image of the target forest region; constructing a multi-stage concentric circular ring search area for the independent closed plaque where the pixel points with the abnormal matching are located; and determining parallax correction parameters of abnormal matching pixel points according to the effective matching pixel points in each level of concentric annulus search region, obtaining all corrected parallax points of the target forest region according to the parallax correction parameters, and reconstructing a forest three-dimensional point cloud model of the target forest region based on all the corrected parallax points. According to the technical scheme provided by the invention, the destructive influence of the matching abnormal points caused by the complex structure of the forest canopy on the three-dimensional reconstruction precision can be overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, more particularly, the present application relates to a high-resolution remote sensing image forest three-dimensional reconstruction method and system. BACKGROUND

[0002] Remote sensing image processing is a key technology for obtaining, analyzing and interpreting information of the earth's surface and atmosphere. In recent years, the popularity of high-resolution satellites (such as sub-meter optical satellites) and synthetic aperture radar (SAR) has led to a huge demand for data processing, driving the iteration of core technologies such as radiation correction, geometric precision correction, and image fusion. At the same time, the introduction of artificial intelligence and deep learning has greatly improved the accuracy of image classification and target detection, providing key technical support for environmental monitoring, urban planning, disaster emergency response and other fields, forming a complete technology chain from data acquisition to intelligent interpretation.

[0003] In existing remote sensing image processing, remote sensing image processing aims to eliminate data errors, enhance effective information, and extract thematic features. First, radiation correction is used to eliminate sensor response bias, atmospheric scattering and other interference, and convert image gray values to real object reflectivity. Geometric correction is based on ground control points, which corrects geometric distortion caused by terrain undulations and sensor attitude through coordinate transformation, achieving precise matching of images and geographic coordinates. However, in high-resolution remote sensing image forest three-dimensional reconstruction, the target forest area often has problems of mutual occlusion of forest canopy leaves and high-frequency texture characteristics. Mutual occlusion of forest canopy leaves and high-frequency texture characteristics can cause non-continuous disparity jumps (i.e., matching abnormalities) during image matching, thereby reducing the geometric accuracy of the forest three-dimensional point cloud model. Therefore, how to overcome the destructive impact of matching abnormal points caused by the complex structure of forest canopy on the accuracy of three-dimensional reconstruction has become a difficult problem in the industry. SUMMARY

[0004] The present application provides a high-resolution remote sensing image forest three-dimensional reconstruction method and system, which can overcome the destructive impact of matching abnormal points caused by the complex structure of forest canopy on the accuracy of three-dimensional reconstruction.

[0005] In a first aspect, the present application provides a high-resolution remote sensing image forest three-dimensional reconstruction method, comprising the following steps: Obtaining a pair of forest remote sensing images of a target forest area, performing pixel-level disparity analysis on the pair of forest remote sensing images to generate a forest disparity map of the target forest area; Locating matching abnormal pixels during remote sensing image matching from the pair of forest remote sensing images through the disparity mapping relationship of the pixels in the forest disparity map; Based on the edge contour continuity of the forest canopy, performing patch segmentation on the forest remote sensing images of the target forest area to obtain independent closed patches with tree crown topological structure as the boundary. For the independent closed patch where the matching abnormal pixel point is located, a multi-level concentric ring search region is constructed with the matching abnormal pixel point as the center in the independent closed patch, wherein the maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumscribed circle of the independent closed patch. The parallax correction parameters of the matching abnormal pixel point in the remote sensing image matching are determined according to the effective matching pixel points in each level of the concentric ring search region, the matching abnormal pixel point is corrected according to the parallax correction parameters, and then all the corrected parallax points of the target forest region are obtained. The forest three-dimensional point cloud model of the target forest region is reconstructed based on all the corrected parallax points.

[0006] In some embodiments, the pixel-level parallax analysis is performed on the pair of forest remote sensing images to generate a forest parallax map of the target forest region, specifically including: The pair of forest remote sensing images is geometrically corrected to obtain a corrected pair of forest remote sensing images; The homonymous feature point pairs are extracted from the corrected pair of forest remote sensing images to form a homonymous feature point pair set; The parallax values of the corresponding pixel points are calculated for each homonymous feature point pair in the homonymous feature point pair set to obtain all the parallax values; The forest parallax map of the target forest region is constructed according to all the parallax values.

[0007] In some embodiments, the matching abnormal pixel point in the remote sensing image matching is located from the pair of forest remote sensing images through the parallax mapping relationship of the pixel points in the forest parallax map, specifically including: The parallax threshold range in the remote sensing image matching process is determined based on the parallax mapping relationship of the pixel points in the forest parallax map; The pixel points with parallax values exceeding the parallax threshold range are screened out from the forest parallax map as candidate abnormal points; The pixel points corresponding to the candidate abnormal points in the pair of forest remote sensing images are taken as the matching abnormal pixel points in the remote sensing image matching.

[0008] In some embodiments, the forest remote sensing image of the target forest region is patch segmented based on the edge contour continuity of the forest canopy, and an independent closed patch with tree crown topological structure as the boundary is obtained, specifically including: The forest remote sensing image of the target forest region is obtained, and the forest remote sensing image is denoised by Gaussian filtering to obtain a smoothed forest remote sensing image; The canopy edge contour map is extracted from the smoothed forest remote sensing image; The morphological connection operation is performed on the canopy edge contour map to obtain the continuous canopy edge contour of the forest canopy; Based on the continuous crown layer edge profile, an initial segmentation is performed on the forest remote sensing image by using a region growing algorithm to obtain initial patches; According to the crown topological structure characteristics, morphological correction is performed on the initial patches to obtain independent closed patches with the crown topological structure as the boundary.

[0009] In some embodiments, for the independent closed patch in which the matching abnormal pixel point is located, a multi-level concentric ring search region is constructed with the matching abnormal pixel point as the center in the independent closed patch, and the multi-level concentric ring search region specifically includes: Determining the minimum circumscribed circle radius of the independent closed patch in which the matching abnormal pixel point is located; Taking the minimum circumscribed circle radius as the maximum search radius of the multi-level concentric ring search region, and setting the radius increment step of the concentric ring; Starting from the initial radius, each level of concentric ring is constructed in turn according to the radius increment step with the matching abnormal pixel point as the center until the maximum search radius is reached; According to the boundary of the independent closed patch, each level of concentric ring is cropped to form a multi-level concentric ring search region in the independent closed patch.

[0010] In some embodiments, the parallax correction parameter of the matching abnormal pixel point in remote sensing image matching is determined according to the effective matching pixel points in each level of concentric ring search region, and the parallax correction parameter specifically includes: From each level of concentric ring search region, the effective matching pixel points are screened out to form a set of effective matching pixel points at each level; The parallax median of the set of effective matching pixel points at each level is calculated to obtain the parallax median at each level; According to the parallax median at each level, a distribution model of the parallax change with the radius is fitted to obtain the parallax distribution model; Based on the parallax distribution model, the predicted parallax value of the matching abnormal pixel point is calculated, and the predicted parallax value is taken as the parallax correction parameter of the matching abnormal pixel point in remote sensing image matching.

[0011] In some embodiments, the parallax of the matching abnormal pixel point is corrected according to the parallax correction parameter, and then all the corrected parallax points of the target forest region are obtained, and the parallax correction specifically includes: The original parallax value corresponding to the matching abnormal pixel point and the parallax correction parameter are extracted; The deviation amount of the original parallax value and the parallax correction parameter is calculated; Based on the deviation amount, the original parallax value of the matching abnormal pixel point is corrected to obtain a corrected parallax value; The corrected parallax value is assigned to the corresponding matching abnormal pixel point, and then all the corrected parallax points of the target forest region are obtained.

[0012] In some embodiments, the pair of forest remote sensing images comprises two remote sensing images of the same forest scene with parallax information.

[0013] In some embodiments, the pair of forest remote sensing images of the target forest region are obtained by synchronously imaging the target forest region by a high-resolution camera.

[0014] In a second aspect, the present application provides a high-resolution remote sensing image forest three-dimensional reconstruction system, comprising: The acquisition module is configured to acquire a pair of forest remote sensing images of a target forest region, perform pixel-level parallax analysis on the pair of forest remote sensing images, and generate a forest parallax map of the target forest region. The processing module is configured to locate matching abnormal pixel points in remote sensing image matching from the pair of forest remote sensing images by a parallax mapping relationship of pixel points in the forest parallax map. The processing module is further configured to perform patch segmentation on the pair of forest remote sensing images of the target forest region based on edge profile continuity of a forest canopy, to obtain independent closed patches with tree crown topological structure as a boundary. The processing module is further configured to, for the independent closed patch in which the matching abnormal pixel points are located, construct a multi-level concentric ring search region with the matching abnormal pixel points as the center in the independent closed patch, wherein a maximum search radius of the multi-level concentric ring search region is constrained by a minimum circumscribed circle of the independent closed patch. The execution module is configured to determine parallax correction parameters of the matching abnormal pixel points in remote sensing image matching according to valid matching pixel points in each concentric ring search region, correct the parallax of the matching abnormal pixel points according to the parallax correction parameters, and then obtain all corrected parallax points of the target forest region, and reconstruct a forest three-dimensional point cloud model of the target forest region based on all the corrected parallax points. The technical scheme provided by the embodiments of the present application has the following beneficial effects: In the high-resolution remote sensing image forest three-dimensional reconstruction method and system provided by the present application, firstly.

[0015] It can be seen that the application can overcome the destructive influence of matching abnormal points caused by the complex structure of the forest canopy on the accuracy of three-dimensional reconstruction. First, a forest disparity map is generated by pixel-level disparity analysis of the forest remote sensing image pair, providing basic disparity data for subsequent abnormal point positioning and three-dimensional reconstruction. Second, the matching abnormal pixel points are located using the disparity mapping relationship, which can accurately identify the error matching area and avoid the interference of the error matching area on the reconstruction accuracy. Further, the independent closed patches are segmented based on the continuity of the canopy edge contour, which can effectively fit the forest boundary in the target forest area and provide an accurate spatial range for targeted processing of matching abnormal pixel points. Then, the multi-level concentric ring search area constrained by the minimum circumscribed circle is constructed within the independent closed patch, which can efficiently extract local effective matching information and improve the reliability of the correction parameters. Finally, the disparity correction parameters are determined based on the effective matching pixel points and the correction is completed, and the forest three-dimensional point cloud model reconstructed based on the corrected disparity points can obtain a forest three-dimensional state with higher detail fidelity and spatial accuracy in a high-density and complex structure forest scene, providing high-quality data support for forest parameter extraction and growth monitoring, thereby avoiding the non-continuous disparity jump caused by image matching when the forest canopy leaves are mutually occluded and the high-frequency texture characteristics. In summary, the technical scheme provided by the application can overcome the destructive influence of matching abnormal points caused by the complex structure of the forest canopy on the accuracy of three-dimensional reconstruction. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is an application scenario architecture schematic diagram of the high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the application; Figure 2 is an exemplary flowchart of the high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the application; Figure 3 is an exemplary flowchart of determining matching abnormal pixel points according to some embodiments of the application; Figure 4 is a structural schematic diagram of a high-resolution remote sensing image forest three-dimensional reconstruction system according to some embodiments of the application; Figure 5 is a structural schematic diagram of a computer device for implementing the high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the application. DETAILED DESCRIPTION

[0017] In order to better understand the technical scheme of the application, the technical scheme of the application will be described in detail below in conjunction with the drawings in the specification and specific embodiments.

[0018] Reference Figure 1FIG. 1 is a schematic diagram of an application scenario architecture of a high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the present application. The application scenario architecture includes a collection terminal, a communication network, and a server end. The collection terminal and the server end are directly or indirectly connected through the communication network. The collection terminal acquires a pair of forest remote sensing images of a target forest region and uploads them to the server end. The server end performs pixel-level disparity analysis on the pair of forest remote sensing images to generate a forest disparity map of the target forest region. The matching abnormal pixel points in remote sensing image matching are located from the pair of forest remote sensing images through the disparity mapping relationship of the pixel points in the forest disparity map. The pair of forest remote sensing images of the target forest region are segmented based on the edge profile continuity of the forest canopy to obtain independent closed patches with tree crown topological structure as the boundary. For the independent closed patch in which the matching abnormal pixel points are located, a multi-level concentric ring search region is constructed with the matching abnormal pixel points as the center in the independent closed patch. The maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumscribed circle of the independent closed patch. The disparity correction parameters of the matching abnormal pixel points in remote sensing image matching are determined according to the valid matching pixel points in each level of the concentric ring search region. The disparity of the matching abnormal pixel points is corrected according to the disparity correction parameters, and then all the corrected disparity points of the target forest region are obtained. The forest three-dimensional point cloud model of the target forest region is reconstructed based on all the corrected disparity points.

[0019] Reference Figure 2 FIG. 2 is an exemplary flowchart of a high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the present application. The high-resolution remote sensing image forest three-dimensional reconstruction method mainly includes the following steps: In step 101, a pair of forest remote sensing images of a target forest region is acquired. Pixel-level disparity analysis is performed on the pair of forest remote sensing images to generate a forest disparity map of the target forest region.

[0020] In implementation, a high-resolution camera is used to synchronously image the target forest region to acquire a pair of forest remote sensing images of the target forest region. The pair of forest remote sensing images contains two remote sensing images with disparity information of the same forest scene.

[0021] It should be noted that the pair of forest remote sensing images of the same target forest region acquired accurately can use the disparity information formed due to the difference in observation angle between the images to provide a geometric reference for subsequent pixel-level disparity analysis, same-name point matching, and three-dimensional coordinate inversion.

[0022] In some embodiments, pixel-level disparity analysis is performed on the pair of forest remote sensing images to generate a forest disparity map of the target forest region, which can be implemented by the following steps, i.e.: Geometric correction is performed on the pair of forest remote sensing images to obtain a corrected pair of forest remote sensing images. extracting a same-name feature point pair from the corrected forest remote sensing image pair, to form a same-name feature point pair set; calculating a parallax value of a corresponding pixel point for each same-name feature point pair in the same-name feature point pair set, to obtain all parallax values; constructing a forest parallax map of the target forest region according to all parallax values.

[0023] In a specific implementation, first, a cubic polynomial model is used to fit the geometric distortion of the forest remote sensing image pair, and then a cubic convolution interpolation method is used to resample the image pixels in the forest remote sensing image pair, to obtain a corrected forest remote sensing image pair, wherein the corrected forest remote sensing image pair represents two remote sensing images in which the geometric distortion is eliminated; second, same-name feature points are extracted from the corrected forest remote sensing image pair, to form a same-name feature point set, that is, a Gaussian pyramid is constructed for each corrected forest remote sensing image in the corrected forest remote sensing image pair, to generate a multi-scale space corresponding to each corrected forest remote sensing image, local extreme points are detected as feature points in different scale spaces, a 128-dimensional feature vector (i.e., a feature descriptor) containing gradient information is established based on the gradient direction of the neighborhood of the feature points, the Euclidean distance of the feature vectors between the feature points in the corrected forest remote sensing image pair is calculated, and a feature point pair with a feature vector Euclidean distance less than a set threshold is taken as a same-name feature point pair, to form a same-name feature point pair set, which refers to a set of pixel points corresponding to the same position in the two corrected forest remote sensing images in the corrected forest remote sensing image pair; third, a forest parallax map of the target forest region is constructed according to all parallax values, that is, a parallax value of a corresponding pixel point is calculated for each same-name feature point pair in the same-name feature point pair set, to obtain all parallax values, which are pixel differences of two pixel points corresponding to the same-name feature point pair; and finally, a forest parallax map of the target forest region is constructed according to all parallax values, that is, the parallax values of all pixels are arranged according to their spatial positions in the remote sensing image, to form a two-dimensional matrix reflecting the parallax distribution, and then a forest parallax map of the target forest region is constructed.

[0024] It should be noted that the forest parallax map of the target forest region in this application represents a set of parallax information of each pixel point in the target forest region in the form of an image, wherein the parallax value represents the horizontal offset of the imaging position of the same position point in the forest remote sensing image pair. In the three-dimensional reconstruction of the forest image, the accurate determination of the forest parallax map can provide reliable basic data support for the subsequent construction of the three-dimensional point cloud model, effectively reduce the parallax error caused by the complex situations such as the shelter of the branches and leaves of the forest canopy, the similarity of the texture, etc., improve the accuracy of the parallax information, provide a more reasonable parallax basis for the subsequent triangular measurement to calculate the three-dimensional coordinates, and then improve the detail expressiveness and overall reconstruction accuracy of the forest three-dimensional point cloud model.

[0025] In step 102, the matching abnormal pixel points in the matching of the remote sensing images are located from the remote sensing image pair through the parallax mapping relationship of the pixel points in the forest parallax map.

[0026] In some embodiments, referring to Figure 3 As shown in the figure, the figure is an exemplary flow chart for determining matching abnormal pixel points according to some embodiments of the present application, and in the present embodiment, the matching abnormal pixel points in the matching of the remote sensing images can be located from the remote sensing image pair through the parallax mapping relationship of the pixel points in the forest parallax map by using the following steps: First, in step 1021, the parallax threshold range in the matching of the remote sensing images is determined based on the parallax mapping relationship of the pixel points in the forest parallax map; Then, in step 1022, the pixel points with parallax values exceeding the parallax threshold range are screened out from the forest parallax map as candidate abnormal points; Finally, in step 1023, the pixel points corresponding to the candidate abnormal points in the remote sensing image pair are taken as the matching abnormal pixel points in the matching of the remote sensing images.

[0027] In the specific implementation, first, the parallax mapping relationship of the pixel points in the forest parallax map is obtained, the parallax mapping relationship is represented by the pixel parallax value, the mean value and the standard deviation of all pixel parallax values in the forest parallax map are calculated by using the statistical analysis method, and then the upper and lower limits of the threshold are set according to the normal distribution principle (for example, the mean value plus or minus 3 times the standard deviation), so as to obtain the parallax threshold range in the matching of the remote sensing images, the parallax threshold range represents the numerical limit for determining whether the parallax value of a pixel point is in a reasonable interval; then, the pixel points with parallax values exceeding the parallax threshold range are screened out from the forest parallax map as candidate abnormal points, the candidate abnormal points refer to the pixel points preliminarily determined as having matching errors; then, the pixel points corresponding to the candidate abnormal points in the remote sensing image pair are taken as the matching abnormal pixel points in the matching of the remote sensing images.

[0028] It should be noted that the matching abnormal pixel points in the present application represent the pixel points with matching errors in the matching of the remote sensing images, through the determination of the matching abnormal pixel points, the error matching area caused by factors such as branch and leaf shielding, repeated texture and light change in the matching of the forest remote sensing images can be accurately identified, an explicit target is provided for subsequent targeted parallax correction, the interference of abnormal values on the accuracy of the overall parallax map is avoided, at the same time, this process can reduce the influence of invalid data on the three-dimensional reconstruction, and ensure that the finally generated forest point cloud model is more consistent with the actual scene characteristics in terms of detail integrity and spatial position accuracy, thereby providing a reliable data basis for subsequent forest parameter extraction, growth state monitoring and other applications.

[0029] In step 103, the forest remote sensing image of the target forest area is patch segmented based on the edge profile continuity of the forest canopy, to obtain independent closed patches with the canopy topological structure as the boundary.

[0030] In some embodiments, the patch segmentation of the forest remote sensing image of the target forest area based on the edge profile continuity of the forest canopy can be achieved by the following steps, that is: Obtaining the forest remote sensing image of the target forest area, and performing Gaussian filtering denoising on the forest remote sensing image to obtain a smoothed forest remote sensing image; Extracting a canopy edge profile map from the smoothed forest remote sensing image; Performing a morphological connection operation on the canopy edge profile map to obtain a continuous canopy edge profile of the forest canopy; Based on the continuous canopy edge profile, performing initial segmentation on the forest remote sensing image by using a region growing algorithm to obtain initial patches; According to the canopy topological structure characteristics, performing morphological correction on the initial patches to obtain independent closed patches with the canopy topological structure as the boundary.

[0031] In a specific implementation, first, a high-resolution camera is used to obtain a forest remote sensing image of a target forest region, and a Gaussian filter is used to denoise the forest remote sensing image to obtain a smoothed forest remote sensing image, where the smoothed forest remote sensing image is a remote sensing image in which main feature information is retained and noise is suppressed. Next, a Canny edge detection algorithm can be used to extract a crown layer edge profile from the smoothed forest remote sensing image, where the crown layer edge profile represents a set of edge lines reflecting the shape of the forest crown layer boundary. Further, a morphological connection operation is performed on the crown layer edge profile to obtain a continuous crown layer edge profile of the forest crown layer, that is, a morphological dilation operation (using a suitable structural element, such as a 3*3 square structure) is used to expand the edge profile in the crown layer edge profile, so that the broken edge segments are connected, and then a corrosion operation is used to restore the original width of the edge to obtain the continuous crown layer edge profile, where the continuous crown layer edge profile is a crown layer profile in which the edge lines are complete and have no obvious breaks after morphological processing. Then, based on the continuous crown layer edge profile, a region growing algorithm is used to perform initial segmentation on the forest remote sensing image to obtain an initial patch, that is, the region growing algorithm is used to take the pixels within the crown layer edge in the continuous crown layer edge profile as seed points, a gray value similarity threshold is set, pixels that are adjacent to the seed points and have a gray value difference within the gray value similarity threshold are merged into the same region, and an initial patch is generated. The gray value similarity threshold can be set according to actual requirements or according to expert knowledge, and is not limited herein. The initial patch is a preliminary segmentation region bounded by the continuous crown layer edge. Finally, morphological correction is performed on the initial patch according to the tree crown topological structure characteristics to obtain an independent closed patch bounded by the tree crown topological structure, that is, the morphological opening operation is used to remove small protrusions on the edge of the initial patch in combination with the fact that the tree crown usually has a circular or elliptical topological structure, and then the morphological closing operation is used to fill small holes in the initial patch to obtain the independent closed patch bounded by the tree crown topological structure.

[0032] It should be noted that the independent closed patch in this application represents a complete closed region in which the target forest region conforms to the natural growth pattern of the tree crown. In this embodiment, the continuity of the forest crown layer edge profile is used as the basis, and morphological operations and tree crown topological feature correction are combined to enable the segmented patch to accurately fit the spatial boundary of a single tree, effectively avoiding the problems of patch adhesion or boundary ambiguity caused by dense growth of trees in traditional methods, improving the accuracy of single tree segmentation, and uniquely using the tree crown topological structure as the core constraint for patch segmentation. Through the cooperative processing of Gaussian filter denoising, Canny edge detection, morphological connection, and region growing, an independent closed unit is constructed that retains the details of the crown layer and conforms to the growth pattern of the forest, providing a precise spatial range for subsequent correction of matching abnormal pixel points for a single tree, and laying a foundation for fine processing of three-dimensional reconstruction.

[0033] In step 104, for the independent closed patch where the matched abnormal pixel point is located, a multi-level concentric ring search region is constructed in the independent closed patch with the matched abnormal pixel point as the center, wherein the maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumscribed circle of the independent closed patch.

[0034] In some embodiments, for the independent closed patch where the matched abnormal pixel point is located, the multi-level concentric ring search region can be constructed in the independent closed patch with the matched abnormal pixel point as the center by the following steps, that is: determining the minimum circumscribed circle radius of the independent closed patch where the matched abnormal pixel point is located; taking the minimum circumscribed circle radius as the maximum search radius of the multi-level concentric ring search region, and setting the radius increment step of the concentric ring; starting from the initial radius, constructing each level of the concentric ring in turn according to the radius increment step with the matched abnormal pixel point as the center, until the maximum search radius is reached; cutting each level of the concentric ring according to the boundary of the independent closed patch to form the multi-level concentric ring search region in the independent closed patch.

[0035] In a specific implementation, first, the minimum circumscribed circle radius of the independent closed patch where the matching abnormal pixel point is located is determined, that is, the rotation puzzle algorithm is used to traverse all pixel points on the boundary of the independent closed patch, the radius of the circle that can completely contain the independent closed patch and has the minimum radius is calculated, and the minimum circumscribed circle radius of the independent closed patch where the matching abnormal pixel point is located is obtained. The minimum circumscribed circle radius refers to the radius of the smallest circle that can surround the independent closed patch. Second, the minimum circumscribed circle radius is used as the maximum search radius of the multi-level concentric ring search area, and the radius increment step of the concentric ring is set. The radius increment step can be set according to the distribution density of the pixel points in the patch or set according to actual needs, which is not limited here. The radius increment step refers to the radius difference between adjacent two levels of concentric rings. Then, starting from the initial radius, each level of concentric ring is constructed in turn according to the radius increment step, until the maximum search radius is reached, that is, the coordinates of the matching abnormal pixel point are taken as the center, starting from the set initial radius (which can be set as the minimum step unit), a circle is drawn each time with an increment of the radius increment step, until the radius of the circle reaches the maximum search radius, and then each level of concentric ring is constructed. Each level of concentric ring refers to a plurality of circular ring bands centered on the matching abnormal pixel point and having radii that increase in turn by the increment step. Finally, the boundary of the independent closed patch is used to crop each level of concentric ring to form a multi-level concentric ring search area in the independent closed patch, that is, the ray method is used to determine whether the pixel points on each level of concentric ring are located inside the independent closed patch, the part located inside the independent closed patch is retained, and the part beyond the boundary of the independent closed patch is removed, and then a multi-level concentric ring search area in the independent closed patch is formed.

[0036] It should be noted that the multi-level concentric ring search area in this application represents the search area formed by each level of concentric ring located inside the independent closed patch. In this embodiment, the minimum circumscribed circle of the independent closed patch is used to constrain the maximum search radius, and the hierarchical ring band is constructed in combination with the radius increment step. This not only avoids the irrelevant pixel interference introduced by the excessively large search range, but also realizes the hierarchical extraction of local effective information through step-by-step expansion, thereby improving the pertinence of abnormal point correction. The patch boundary is used as the natural constraint of the search area, and the search range is strictly limited in the single tree crown through the ray method cropping. At the same time, the multi-level ring band structure is used to capture the parallax distribution law of the neighborhood at different distances, so that the calculation of the subsequent parallax correction parameter is more in line with the local morphological characteristics of the tree crown. Especially in a complex crown scene, the search accuracy and calculation efficiency can be effectively balanced, and the structured neighborhood information support is provided for accurate correction of the matching abnormal pixel point.

[0037] In step 105, the parallax correction parameters of the matching abnormal pixel points in the remote sensing image matching are determined according to the effective matching pixel points in the search area of each concentric circle ring, the parallax correction parameters are used for parallax correction of the matching abnormal pixel points, and then all the corrected parallax points of the target forest region are obtained, and a three-dimensional point cloud model of the forest region of the target forest region is reconstructed based on all the corrected parallax points.

[0038] In some embodiments, the parallax correction parameters of the matching abnormal pixel points in the remote sensing image matching can be determined according to the effective matching pixel points in the search area of each concentric circle ring, and the following steps can be used to achieve the determination, that is: The effective matching pixel points are screened from the search area of each concentric circle ring to form a set of effective matching pixel points of each level; The parallax median of the set of effective matching pixel points of each level is calculated to obtain the parallax median of each level; A distribution model of the parallax changing with the radius is fitted according to the parallax medians of each level to obtain the parallax distribution model; The predicted parallax value of the matching abnormal pixel points is calculated based on the parallax distribution model, and the predicted parallax value is used as the parallax correction parameter of the matching abnormal pixel points in the remote sensing image matching.

[0039] In a specific implementation, first, valid matching pixel points are screened from each level of the concentric circular ring search area to form a set of valid matching pixel points for each level, that is, for each level of the concentric circular ring search area, the standard deviation and mean value of all disparity values in the concentric circular ring search area are calculated, the pixel points with disparity values within the range of the mean value plus or minus 3 times the standard deviation are regarded as valid matching pixel points, and all the valid matching pixel points are combined into a set of valid matching pixel points, and then valid matching pixel points are screened from each level of the concentric circular ring search area to form a set of valid matching pixel points for each level, which refers to a set of pixel points with stable disparity in each level of the concentric circular ring search area; second, the disparity values of the pixel points in each set of valid matching pixel points are sorted in descending order, and the disparity value at the middle position is taken as the median disparity of the corresponding level to obtain the median disparity of each level, which refers to the disparity value at the middle position after the disparity values of each set of valid matching pixel points are sorted in descending order, and is used to measure the central tendency of the disparity values in the set of valid matching pixel points; then, a distribution model of the change of disparity with radius is fitted according to the median disparity of each level to obtain the disparity distribution model, that is, the radius of each level of the concentric circular ring is taken as the independent variable, and the corresponding median disparity is taken as the dependent variable, and a polynomial fitting (such as a quadratic polynomial) is performed by using the least square method to obtain a function model that can reflect the change law of the disparity with the radius, that is, the disparity distribution model, which refers to a function model that can reflect the change law of the disparity with the radius of the concentric circular ring; finally, the predicted disparity value of the matching abnormal pixel point is calculated based on the disparity distribution model, and the predicted disparity value is taken as the disparity correction parameter of the matching abnormal pixel point in the matching of the remote sensing image, that is, the distance between the matching abnormal pixel point and the nearest matching normal pixel point is taken as the output radius, which is substituted into the disparity distribution model to obtain the function value as the predicted disparity value of the matching abnormal pixel point, and the predicted disparity value is taken as the disparity correction parameter of the matching abnormal pixel point in the matching of the remote sensing image, which refers to the predicted disparity value used to correct the disparity of the matching abnormal pixel point.

[0040] It should be noted that the disparity correction parameter in this application represents the parameter value used to correct the disparity of the matching abnormal pixel point, and in this embodiment, valid matching pixel points are screened by multi-level concentric circular ring layering, combined with the robust statistical characteristics of the median disparity, to reduce the interference of local noise on parameter estimation, and a polynomial fitting is used to construct a distribution model of the change of disparity with radius, so that the correction parameter can accurately reflect the local disparity change law of the forest canopy, and the accuracy of the abnormal point correction is improved. The spatial hierarchical structure of the concentric circular ring is combined with the radius distribution characteristics of the disparity, and the model fitting based on the median of each level retains the gradient change of the neighborhood information, so that the correction parameter is more suitable for the three-dimensional morphological characteristics of the forest, and provides key support for the detail fidelity of subsequent point cloud reconstruction.

[0041] In some embodiments, the disparity correction parameter is used to correct the matching abnormal pixel points, and all corrected disparity points of the target forest region are obtained by the following steps: extracting the original disparity value and the disparity correction parameter corresponding to the matching abnormal pixel points; calculating the deviation of the original disparity value and the disparity correction parameter; correcting the original disparity value of the matching abnormal pixel points based on the deviation to obtain a corrected disparity value; assigning the corrected disparity value to the corresponding matching abnormal pixel points, and then obtaining all corrected disparity points of the target forest region.

[0042] In a specific implementation, first, the matching abnormal pixel points marked in the forest disparity map are traversed, and the original disparity value and the disparity correction parameter corresponding to the matching abnormal pixel points in the forest disparity map are read. The original disparity value refers to the disparity value of the matching abnormal pixel points before correction. Second, when calculating the deviation of the original disparity value and the disparity correction parameter, the original disparity value is subtracted from the disparity correction parameter by using subtraction operation to obtain the deviation of the original disparity value and the disparity correction parameter. The deviation refers to the difference between the original disparity value and the disparity correction parameter. Third, when correcting the original disparity value of the matching abnormal pixel points based on the deviation to obtain a corrected disparity value, the corrected disparity value is obtained by subtracting the deviation from the original disparity value. The corrected disparity value refers to the disparity value of the matching abnormal pixel points after correction by the disparity correction parameter. Finally, the corrected disparity value is assigned to the corresponding matching abnormal pixel points, and then all corrected disparity points of the target forest region are obtained. That is, the original disparity value of the corresponding matching abnormal pixel points is replaced by the calculated corrected disparity value, while the disparity values of other normally matched pixel points in the forest disparity map are retained, and all corrected disparity points of the target forest region are obtained.

[0043] It should be noted that all corrected disparity points in this application refer to a set of corrected matching abnormal pixel points and normally matched pixel points in the target forest region. By determining the corrected disparity points, the disparity error of the matching abnormal pixel points can be eliminated, the disparity information is more consistent with the actual spatial distribution characteristics of the forest, and accurate basic data are provided for three-dimensional point cloud reconstruction. At the same time, this process integrates the disparity information of the normally matched pixel points and the corrected pixel points, ensures the completeness and reliability of the disparity data of the target forest region, and further improves the accuracy of subsequent triangular measurement of three-dimensional coordinates. In some embodiments, the forest three-dimensional point cloud model of the target forest region is reconstructed based on all corrected disparity points by the following steps: obtaining the intrinsic and extrinsic parameters of the forest remote sensing image pair; According to the intrinsic parameters, the extrinsic parameters and all the corrected parallax points, three-dimensional coordinates of each corrected parallax point are calculated through the principle of triangulation; The three-dimensional coordinates of all the corrected parallax points are converted into a preset coordinate system to form a three-dimensional point cloud; A three-dimensional point cloud model of the target forest area is constructed based on the three-dimensional point cloud.

[0044] In a specific implementation, first, when the intrinsic parameters and the extrinsic parameters of the forest remote sensing image pair are obtained, the intrinsic parameters (including focal length, principal point coordinates and distortion coefficients) of the camera are calculated through a camera calibration method (such as Zhang Zhengyou calibration method) by using the captured checkerboard calibration board image, and then the rotation matrix and the translation vector of the forest remote sensing image pair are solved as the extrinsic parameters through a relative orientation method, wherein the intrinsic parameters refer to parameters describing the optical characteristics and imaging geometric relationship of the camera itself, and the extrinsic parameters refer to parameters describing the relative position and attitude of the two images in space; second, according to the intrinsic parameters, the extrinsic parameters and all the corrected parallax points, three-dimensional coordinates of each corrected parallax point are calculated through the principle of triangulation, that is, the pixel coordinates of the corrected parallax points in the forest remote sensing image pair are respectively substituted into the camera imaging model, the spatial geometric relationship is constructed in combination with the intrinsic parameters and the extrinsic parameters, and the three-dimensional coordinates of the corrected parallax points are solved according to the parallax information of the corresponding points in the forest remote sensing image pair, wherein the three-dimensional coordinates refer to the corresponding position coordinates of each corrected parallax point in the three-dimensional space; third, a coordinate transformation matrix (solved through an absolute orientation method) is used to convert the three-dimensional coordinates of each corrected parallax point from the image coordinate system to a preset coordinate system (i.e. a geodetic coordinate system) to form a three-dimensional point cloud, wherein the three-dimensional point cloud refers to a point set composed of three-dimensional coordinate points, reflecting the spatial form of the target forest area; and finally, a three-dimensional point cloud model of the target forest area is constructed based on the three-dimensional point cloud, that is, the three-dimensional point cloud is denoised (such as using statistical filtering to remove outliers), and the coordinate points reflecting the characteristics of the forest canopy and branches are retained, and then the three-dimensional point cloud model of the target forest area is obtained.

[0045] It should be noted that the three-dimensional point cloud model of the target forest area in the present application refers to a three-dimensional point cloud set that can completely present the spatial structure and form characteristics of the target forest area. Through determination of the three-dimensional point cloud model of the forest, two-dimensional remote sensing image information can be converted into three-dimensional spatial structure data, and the canopy form, branch distribution and spatial position relationship of single trees in the target forest area can be intuitively and accurately presented, thereby providing quantifiable three-dimensional parameters for forest resource investigation.

[0046] In addition, another aspect of the present application provides a high-resolution remote sensing image forest three-dimensional reconstruction system in some embodiments, which is described with reference to Figure 4FIG. 1 is a structural schematic diagram of a high-resolution remote sensing image forest three-dimensional reconstruction system according to some embodiments of the present application, which includes an acquisition module 201, a processing module 202, and an execution module 203, which are described as follows: The acquisition module 201 is mainly used for acquiring a pair of forest remote sensing images of a target forest region in the present application, performing pixel-level disparity analysis on the pair of forest remote sensing images to generate a forest disparity map of the target forest region. The processing module 202 is mainly used for locating matching abnormal pixel points in remote sensing image matching from the pair of forest remote sensing images based on the disparity mapping relationship of the pixel points in the forest disparity map in the present application. The processing module 202 is also used for performing patch segmentation on the forest remote sensing images of the target forest region based on the edge contour continuity of the forest canopy to obtain independent closed patches with tree crown topological structure as the boundary. In addition, the processing module 202 is also used for constructing a multi-level concentric ring search region with the matching abnormal pixel points as the center in the independent closed patch where the matching abnormal pixel points are located, wherein the maximum search radius of the multi-level concentric ring search region is constrained by the minimum circumscribed circle of the independent closed patch. The execution module 203 is mainly used for determining the disparity correction parameters of the matching abnormal pixel points in remote sensing image matching according to the valid matching pixel points in each level of the concentric ring search region, performing disparity correction on the matching abnormal pixel points according to the disparity correction parameters, and then obtaining all the corrected disparity points of the target forest region, and reconstructing the forest three-dimensional point cloud model of the target forest region based on all the corrected disparity points.

[0047] In addition, the present application also provides a computer device, which includes a memory and a processor, the memory stores code, and the processor is configured to acquire the code and execute the high-resolution remote sensing image forest three-dimensional reconstruction method described above.

[0048] In some embodiments, with reference to Figure 5 FIG. 1 is a structural schematic diagram of a computer device for implementing the high-resolution remote sensing image forest three-dimensional reconstruction method according to some embodiments of the present application. The high-resolution remote sensing image forest three-dimensional reconstruction method in the above embodiments can be implemented by the computer device shown in FIG. 1, which includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304. Figure 5

[0049] ​The processor 301 can be a general central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the high-resolution remote sensing image forest three-dimensional reconstruction method in the application.

[0050] The communication bus 302 can be used to transmit information between the above-mentioned components.

[0051] The memory 303 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited to this. The memory 303 can exist independently and be connected to the processor 301 through the communication bus 302. The memory 303 can also be integrated with the processor 301.

[0052] The memory 303 is used to store program code for executing the scheme of the application, and the processor 301 is used to control the execution. The processor 301 is used to execute the program code stored in the memory 303. The program code can include one or more software modules. The determination of the high-resolution remote sensing image forest three-dimensional reconstruction method in the above-mentioned embodiments can be realized by the processor 301 and one or more software modules in the program code in the memory 303.

[0053] The communication interface 304 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0054] In a particular implementation, as one embodiment, the computer device can include multiple processors, each of which can be a single-CPU processor or a multi-CPU processor. A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0055] The computer device described above can be a general-purpose computer device or a special-purpose computer device. In a particular implementation, the computer device can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0056] In addition, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the high-resolution remote sensing image forest three-dimensional reconstruction method.

[0057] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all the preferred embodiments and all the changes and modifications falling within the scope of the present application.

[0058] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A high-resolution remote sensing image forest three-dimensional reconstruction method, characterized in that, The method comprises the following steps: Obtaining a pair of forest remote sensing images of a target forest area, performing pixel-level disparity analysis on the pair of forest remote sensing images, and generating a forest disparity map of the target forest area; Locating matching abnormal pixel points in remote sensing image matching from the pair of forest remote sensing images through a disparity mapping relationship of pixel points in the forest disparity map; Performing patch segmentation on the pair of forest remote sensing images of the target forest area based on the edge profile continuity of the forest canopy, and obtaining independent closed patches with tree crown topological structure as boundaries; For the independent closed patch in which the matching abnormal pixel points are located, a multi-level concentric ring search area is constructed with the matching abnormal pixel points as the center in the independent closed patch, wherein the maximum search radius of the multi-level concentric ring search area is constrained by the minimum circumscribed circle of the independent closed patch; Determining disparity correction parameters of the matching abnormal pixel points in remote sensing image matching according to valid matching pixel points in each level of the concentric ring search area, performing disparity correction on the matching abnormal pixel points according to the disparity correction parameters, and then obtaining all corrected disparity points of the target forest area, and reconstructing a forest three-dimensional point cloud model of the target forest area based on all the corrected disparity points.

2. The method of claim 1, wherein, The pixel-level disparity analysis on the pair of forest remote sensing images to generate the forest disparity map of the target forest area specifically comprises: Performing geometric correction on the pair of forest remote sensing images to obtain a pair of corrected forest remote sensing images; Extracting corresponding feature point pairs from the pair of corrected forest remote sensing images to form a corresponding feature point pair set; Calculating the disparity value of the corresponding pixel points for each corresponding feature point pair in the corresponding feature point pair set to obtain all disparity values; Constructing the forest disparity map of the target forest area according to all the disparity values.

3. The method of claim 1, wherein, The locating of the matching abnormal pixel points in remote sensing image matching from the pair of forest remote sensing images through the disparity mapping relationship of pixel points in the forest disparity map specifically comprises: Determining a disparity threshold range in the remote sensing image matching process based on the disparity mapping relationship of pixel points in the forest disparity map; Screening pixel points with disparity values exceeding the disparity threshold range from the forest disparity map as candidate abnormal points; Taking the pixel points corresponding to the candidate abnormal points in the pair of forest remote sensing images as matching abnormal pixel points in remote sensing image matching.

4. The method of claim 1, wherein, The patch segmentation on the pair of forest remote sensing images of the target forest area based on the edge profile continuity of the forest canopy specifically comprises: Obtaining the pair of forest remote sensing images of the target forest area, and performing Gaussian filter denoising on the pair of forest remote sensing images to obtain a smoothed pair of forest remote sensing images; Extracting a canopy edge profile map from the smoothed pair of forest remote sensing images; Performing a morphological connection operation on the canopy edge profile map to obtain a continuous canopy edge profile of the forest canopy; Performing initial segmentation on the pair of forest remote sensing images based on the continuous canopy edge profile by using a region growing algorithm to obtain initial patches; Performing morphological correction on the initial patches according to the tree crown topological structure characteristics to obtain independent closed patches with tree crown topological structure as boundaries.

5. The method of claim 1, wherein, The method comprises the following steps: determining the minimum circumscribed circle radius of the independent closed patch where the matching abnormal pixel point is located; taking the minimum circumscribed circle radius as the maximum search radius of the multi-level concentric ring search area, and setting the radius increment step of the concentric ring; starting from the initial radius, each level of the concentric ring is constructed in turn according to the radius increment step, until the maximum search radius is reached; the multi-level concentric ring search area in the independent closed patch is formed by clipping each level of the concentric ring according to the boundary of the independent closed patch.

6. The method of claim 1, wherein, The method comprises the following steps: screening the effective matching pixel points from each level of the concentric ring search area to form a set of effective matching pixel points of each level; calculating the parallax median of the set of effective matching pixel points of each level to obtain the parallax median of each level; fitting a distribution model of the parallax changing with the radius according to the parallax median of each level to obtain the parallax distribution model; calculating the predicted parallax value of the matching abnormal pixel point based on the parallax distribution model, and taking the predicted parallax value as the parallax correction parameter of the matching abnormal pixel point in the remote sensing image matching.

7. The method of claim 1, wherein, The method comprises the following steps: extracting the original parallax value and the parallax correction parameter corresponding to the matching abnormal pixel point; calculating the deviation amount of the original parallax value and the parallax correction parameter; correcting the original parallax value of the matching abnormal pixel point based on the deviation amount to obtain the corrected parallax value; assigning the corrected parallax value to the corresponding matching abnormal pixel point to obtain all the corrected parallax points of the target forest region.

8. The method of claim 1, wherein, The pair of forest remote sensing images comprises two remote sensing images containing the same forest scene and having parallax information.

9. The method of claim 1, wherein, The pair of forest remote sensing images of the target forest region is obtained by synchronously imaging the target forest region through a high-resolution camera.

10. A high-resolution remote sensing image forest three-dimensional reconstruction system, characterized in that, The system comprises: an acquisition module configured to acquire the pair of forest remote sensing images of the target forest region, perform pixel-level parallax analysis on the pair of forest remote sensing images, and generate a forest parallax map of the target forest region; a processing module configured to locate the matching abnormal pixel point in the remote sensing image matching through the parallax mapping relationship of the pixel points in the forest parallax map; the processing module is further configured to perform patch segmentation on the pair of forest remote sensing images of the target forest region based on the edge contour continuity of the forest canopy to obtain independent closed patches with the tree crown topological structure as the boundary; the processing module is further configured to construct a multi-level concentric ring search area in the independent closed patch where the matching abnormal pixel point is located, with the matching abnormal pixel point as the center, wherein the maximum search radius of the multi-level concentric ring search area is constrained by the minimum circumscribed circle of the independent closed patch. The execution module is used for determining the parallax correction parameter of the matching abnormal pixel point in the remote sensing image matching according to the valid matching pixel points in the search area of each concentric circle, correcting the parallax of the matching abnormal pixel point according to the parallax correction parameter, and then obtaining all the corrected parallax points of the target forest region, and reconstructing the forest three-dimensional point cloud model of the target forest region based on all the corrected parallax points.

Images