Deep learning-based vegetation feature model reconstruction method and system

Abstract

The application belongs to the technical field of surveying and mapping, and provides a vegetation ground object model reconstruction method based on deep learning. The method adopts the deep learning method to identify the vegetation area in the target scene and the oblique image, and applies the identification result to the subsequent multi-view image three-dimensional reconstruction process, so as to improve the reconstruction efficiency of the vegetation ground object model, and improve the visualization efficiency of the three-dimensional scene.

Description

Technical Field

[0001] The present invention belongs to the field of surveying and mapping technology, and mainly relates to a method and system for reconstructing vegetation landform models based on deep learning. Background Art

[0002] In recent years, with the rapid development of oblique photogrammetry technology and image acquisition equipment, 3D reconstruction methods based on multi-view images have been widely used in the 3D reconstruction of large-scale scenes. Although 3D reconstruction methods based on oblique photography have the advantages of low cost and high degree of automation, there are still some problems that need to be solved in actual 3D modeling applications. For example, vegetation target objects such as grass, nurseries, and trees in the scene are of low importance in most 3D scene applications. However, due to their rich shape and structural characteristics, the number of triangles in this area in the generated 3D model is relatively large, which will seriously affect the production efficiency and visualization efficiency of the 3D scene model. Summary of the Invention

[0003] The present invention provides a vegetation object model reconstruction method based on deep learning, aiming to improve the reconstruction efficiency and visualization efficiency of vegetation object models in three-dimensional scenes.

[0004] To achieve the above objectives, the present invention provides a method for reconstructing a vegetation model based on deep learning, the method comprising:

[0005] The vegetation features in the oblique image are identified using deep learning methods, and the identification results of the vegetation feature areas in the reconstructed scene are obtained based on the image recognition results;

[0006] Performing image screening on the recognition result of the vegetation feature area using an image screening method integrated with feature classification to obtain a screened image;

[0007] Performing dense matching on the filtered images to obtain dense point cloud data of the reconstructed scene, and processing the dense point cloud data using a thinning and resampling method to obtain a sampled dense point cloud;

[0008] The sampled dense point cloud is constructed using a triangular mesh method to obtain a three-dimensional surface mesh of the target scene, the three-dimensional surface mesh is simplified using different strategies to obtain a simplified three-dimensional surface mesh model, and texture mapping is performed based on the simplified three-dimensional surface mesh model to obtain the final three-dimensional model of the target scene.

[0009] Furthermore, the method of identifying vegetation features in the oblique image using a deep learning method and obtaining a vegetation feature area in the reconstructed scene based on the image recognition result includes:

[0010] Using the UNet network in deep learning, pixel-level semantic segmentation is performed on the vegetation objects in the oblique image to determine the vegetation object area in the oblique image;

[0011] Projecting each 3D connection point in the reconstructed scene onto all its observation images according to spatial triangulation to obtain observation image information corresponding to each 3D connection point;

[0012] According to the observation image information corresponding to each three-dimensional connection point, the ground object category of the three-dimensional connection point is determined, and the recognition result of the vegetation ground object area in the reconstructed scene is obtained.

[0013] Furthermore, the UNet network in deep learning is used to perform pixel-level semantic segmentation on the vegetation features in the oblique image to determine the vegetation feature area in the oblique image, including:

[0014] Downsampling the image corresponding to the reconstructed scene through an encoder in the UNet network body, and upsampling the image corresponding to the reconstructed scene through a decoder in the UNet network body;

[0015] Through the upsampling results, downsampling results and the output layer in the UNet network body, the probability that each pixel in the image of the reconstructed scene belongs to each ground object category is output, and the vegetation ground object area in the image of the reconstructed scene is determined based on the probability that each pixel belongs to each ground object category.

[0016] Furthermore, determining the feature category of each 3D connection point based on the observed image information corresponding to the 3D connection point, and obtaining the recognition result of the vegetation feature area in the reconstructed scene, includes:

[0017] Projecting each 3D tie point in the scene onto all its observation images according to the observation image information corresponding to each 3D tie point, and counting the ground feature categories to which pixels at the projection positions of the 3D tie point on different observation images belong;

[0018] The ground feature category with the largest number is determined as the ground feature category to which the three-dimensional connection point belongs, and the recognition result of the vegetation feature area is obtained.

[0019] Furthermore, the identification result of the vegetation feature area is subjected to image screening using an image screening method integrating feature classification to obtain the screened image, comprising:

[0020] Obtaining a set of candidate visual observation images in the reconstructed scene using observation image information corresponding to each three-dimensional connection point;

[0021] A greedy method is used to select the same number of images from the candidate visual observation image set as the screened images.

[0022] Furthermore, the obtaining of a set of candidate visible observation images in the reconstructed scene by utilizing the observation image information corresponding to each three-dimensional connection point includes:

[0023] According to the scope of the three-dimensional outer box of the reconstructed scene, the reconstructed scene is divided into a preset number of small space cubes according to a preset spacing;

[0024] According to the three-dimensional spatial coordinates of the three-dimensional connection points, counting the point set of the three-dimensional connection points falling into each cube;

[0025] According to the observation image information corresponding to each three-dimensional connection point in the point set, a candidate visual observation image set of each cube is counted.

[0026] Furthermore, the method of selecting the same number of images from the candidate visual observation image set using a greedy method as the filtered images includes:

[0027] Using the observation image information corresponding to each three-dimensional connection point in the point set, a voting method is used to calculate the score of each visible image in the candidate visible observation image set of the cube;

[0028] An upper limit on the number of images that can be selected from the cube is set according to a preset number of images in the candidate visual observation image set of the cube and a preset formula, wherein the preset formula is:

[0029]

[0030] Filtering, according to the score of each visible image in the candidate visible observation image set of the cube, cubes whose number of currently selectable images does not exceed the upper limit of the number of images;

[0031] Counting the image with the largest score for each visible image in the cube whose number of filtered images does not exceed the upper limit of the number of images, taking it as the current best image, and setting the best image as the selected image;

[0032] increasing the number of images selected by the cube including the best image in all the candidate visual observation image sets by 1, and deleting the best image from the candidate visual observation image set of the cube;

[0033] Repeat the process: add 1 to the number of images selected by the cubes that contain the best image in all the candidate visual observation image sets, and delete the best image from the candidate visual observation image sets of the cubes, until the number of images selected by each cube reaches the maximum upper limit of the number of images set, and output all selected images as the filtered images.

[0034] Furthermore, the dense point cloud data is processed using a thinning and resampling method to obtain a sampled dense point cloud, comprising:

[0035] Dividing the reconstructed scene into a preset number of small spatial cubes according to the three-dimensional space bounding box range of the dense point cloud and at a preset spacing, and counting the point sets of dense matching points falling into each cube;

[0036] According to the feature category of each dense matching point in the point set, the feature category with the largest number is used as the feature category of the corresponding cube;

[0037] Select a cube with vegetation as the ground feature category, merge all dense matching points in the point set of the vegetation cube, and obtain all merged vegetation points in the reconstructed scene, where the coordinates of the merged vegetation points are the average of the coordinates of all points in the point set of each cube;

[0038] The dense points in the cubes where all objects are classified as non-vegetation and all the merged vegetation points are taken as the dense point cloud after sampling.

[0039] Furthermore, the three-dimensional surface mesh is simplified using different strategies to obtain a simplified three-dimensional surface mesh model, including:

[0040] For the three-dimensional surface mesh of non-vegetated areas, the first level of simplification is used;

[0041] A second level of simplification strength is applied to the three-dimensional surface mesh of the vegetation area to obtain a simplified model with less triangular face data, wherein the second level is greater than the first level.

[0042] To achieve the above objectives, the present invention further provides a vegetation model reconstruction system based on deep learning, the system comprising:

[0043] The image recognition module is used to identify vegetation objects in oblique images using deep learning methods, and obtain the vegetation object area in the reconstructed scene based on the image recognition results;

[0044] An image screening module is used to screen the recognition results of the vegetation feature area using an image screening method integrated with feature classification to obtain a screened image;

[0045] An image matching module is used to perform dense matching on the filtered images to obtain dense point cloud data of the reconstructed scene, and to process the dense point cloud data using a thinning and resampling method to obtain a sampled dense point cloud;

[0046] The model reconstruction module is used to construct the sampled dense point cloud using a triangular mesh method to obtain a three-dimensional surface mesh of the target scene, simplify the three-dimensional surface mesh using different strategies to obtain a simplified three-dimensional surface mesh model, and perform texture mapping based on the simplified three-dimensional surface mesh model to obtain the final three-dimensional model of the target scene.

[0047] The present invention proposes a method for reconstructing vegetation models based on deep learning. The method uses a deep learning method to identify vegetation areas in oblique images and target scenes, filters and resamples the identification results to obtain sampled dense point cloud data, constructs a three-dimensional surface mesh model for the sampled dense point cloud data, simplifies the three-dimensional surface mesh model and applies it to the subsequent multi-view image three-dimensional reconstruction process to achieve the purpose of improving the efficiency of reconstructing vegetation models. The method proposed in the present invention can improve the reconstruction efficiency of three-dimensional scenes by 5% to 49% according to the proportion of different vegetation areas in the reconstructed scene. It improves the efficiency of three-dimensional scene reconstruction and visualization, and has a high degree of automation, without the need for human intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 This is a flowchart of a method for reconstructing a vegetation model based on deep learning provided by an embodiment of the present application;

[0050] Figure 2 This is a result diagram of identifying vegetation and ground objects in oblique images using a deep learning method provided by an embodiment of the present application;

[0051] Figure 3 This is a statistical chart showing the improvement in three-dimensional reconstruction efficiency under different vegetation proportions in a scene of a vegetation feature model reconstruction method based on deep learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and technical effect of the present invention more clearly understood, the specific embodiments of the present invention are described below. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0054] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0055] It should be further understood that the term “and / or” used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0056] The embodiments of the present application provide a method for reconstructing a vegetation model based on deep learning. This method can be applied to the field of surveying and mapping technology. It uses a deep learning method to identify vegetation areas in oblique images and reconstructed scene images, and filters and resamples the identification results to obtain sampled dense point cloud data. A three-dimensional surface mesh model is constructed for the sampled dense point cloud data. The simplified three-dimensional surface mesh model is then applied to the subsequent multi-view image three-dimensional reconstruction process to improve the reconstruction of the vegetation model and enhance the visualization efficiency of the three-dimensional scene.

[0057] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0058] See also Figure 1 , Figure 1 This is a flowchart of a method for reconstructing a vegetation model based on deep learning provided in an embodiment of the present application.

[0059] S10, identifying vegetation features in the oblique image using a deep learning method, and obtaining an identification result of the vegetation feature area in the reconstructed scene based on the image recognition result;

[0060] Traditional object recognition algorithms are mostly based on template and feature matching. However, as scene complexity increases, these artificially generated templates and features experience significant performance degradation and mismatches. Advances in deep learning technology have demonstrated strong robustness and interference resistance in content recognition, classification, and feature extraction. Various methods are now available for identifying vegetation areas.

[0061] In this embodiment, the deep learning UNet network (Huang et al., 2020) is used as a convolutional neural network to quickly and accurately extract the vegetation area in the reconstructed scene image to obtain the vegetation feature area in the reconstructed scene image.

[0062] In this embodiment, in order to further determine the types of objects (vegetation or non-vegetation) in different areas of the three-dimensional scene, the observation image information corresponding to the connection points in the aerial triangulation (AT) results is used for determination.

[0063] Furthermore, the UNet network in deep learning is used to perform pixel-level semantic segmentation on the vegetation objects in the oblique image to determine the vegetation object area in the oblique image, which specifically includes:

[0064] The main body of the UNet network is divided into an encoder and a decoder: the encoder uses a convolution kernel of 3, a batch normalization layer BatchNormlize, and an activation function ReLU as the basic unit, and then concatenates them to downsample the image step by step;

[0065] The decoder uses deconvolution with a convolution kernel of 3, batch normalization layer BatchNormlize and activation function ReLU as basic units to perform step-by-step upsampling;

[0066] The output layer is connected to a Sigmoid activation function to output the probability that each pixel belongs to a different category. The recognition results are as follows: Figure 2 shown.

[0067] Furthermore, determining the feature category of each 3D connection point based on the observed image information corresponding to the 3D connection point, and obtaining the recognition result of the vegetation feature area in the reconstructed scene, specifically includes:

[0068] We connect each 3D point p in the scene i Projected onto all its observation images respectively;

[0069] Statistical connection point p i The ground feature category to which the pixel at its projected position on different observation images belongs;

[0070] According to the principle of "winner takes all", the category with the largest number of items is designated as the current connection point p i The feature category.

[0071] Through the processing of the above method, we can obtain the object category to which each three-dimensional connection point in the scene belongs, that is, the recognition result of the vegetation object area in the reconstructed scene.

[0072] S20, performing image screening on the recognition result of the vegetation feature area using an image screening method integrated with feature classification to obtain a screened image;

[0073] Image matching uses the image recognition results of vegetation feature areas for dense image matching. Dense image matching is the most important method for obtaining dense point clouds in multi-view 3D reconstruction. However, for oblique photography, the image data collected by the oblique photography platform often contains a lot of redundancy. Therefore, in this embodiment, the images are screened before performing dense image matching to improve the efficiency of 3D reconstruction. Currently, many scholars have conducted research on this issue, among which the CMVS method (Furukawa et al., 2010) is the most representative. However, algorithms such as CMVS are indiscriminate in the image screening process and do not consider the distinguishability of the reconstructed scene content. Using an image screening method that integrates feature classification, different degrees of image screening are performed according to different feature types. The filtered images are then used for dense image matching.

[0074] In this implementation case, the observation image information corresponding to each 3D connection point in the reconstructed scene is used to obtain a set of candidate visual observation images in the reconstructed scene. In order to ensure that the selected image can completely cover the 3D scene to be reconstructed, it is necessary to select a candidate image from each small cube c. i Select a certain number (n sel ) images; and to minimize the number of selected images, it is necessary to ensure that the images selected between adjacent small cubes are the same. Based on the importance of different landform types in the scene (vegetation areas are often less important), a "greedy method" is used to select images as the filtered images.

[0075] Furthermore, the obtaining of a set of candidate visual observation images in the reconstructed scene using the observation image information corresponding to each three-dimensional connection point specifically includes:

[0076] According to the three-dimensional space bounding box range of the scene and the preset spacing, the reconstructed scene is divided into a preset number of small space cubes C = {c1, c2, ..., c n};

[0077] According to the three-dimensional space coordinates of the connection points in the AT results, each cube c is countedi The point set P(c i )={p1,p2,…,p m};

[0078] According to the point set P(c i ) corresponds to the observation image information of each three-dimensional connection point, and statistics are obtained for each cube c i The candidate visual observation image set I(c i )={I1,I2,…,I k}.

[0079] Furthermore, the method of selecting the same number of images from the candidate visual observation image set using the greedy method as the screened images specifically includes:

[0080] According to the cube c i The point set P(c i ) is the category with the largest number of ground feature categories (vegetation or non-vegetation) at each connection point as the current cube c i The type of land features;

[0081] Using the point set P(c i ) is the visual image information statistics cube c of each connection point i The candidate image set I(c i ) is the score of each visual image in the image. Here, a voting method is used, that is, the score of each visual image corresponding to the aerotriangulated point increases by 1 each time it appears.

[0082] According to the cube c i The candidate image set I(c i ) is set using the following formula: i The maximum number of images that can be selected in:

[0083] According to c i The fraction of visible images in the filter is used to filter out the currently selected images whose number does not exceed the upper limit n. sel The cube c i , the image with the largest score of visible candidate images in these cubes is counted as the current best image I best , and set I best For the selected image;

[0084] All candidate visual images contain I best The cube c i The number of selected images increases by 1, and I best Remove from these cubes candidate visual images;

[0085] Repeat the above process until each cube c i The number of selected images has reached the maximum limit n sel So far, all selected images are output as filtered images.

[0086] S30, performing dense matching on the filtered images to obtain dense point cloud data of the reconstructed scene, and processing the dense point cloud data using a thinning and resampling method to obtain the sampled dense point cloud specifically includes:

[0087] After dense matching using the filtered images, dense point cloud data for the reconstructed scene is obtained. Considering that most 3D scene applications do not require high-quality modeling of vegetation and ground objects, in order to improve the efficiency of triangulated mesh construction from dense point clouds, the vegetation and ground object areas in the dense point cloud are thinned and resampled to obtain a dense point cloud. This is then used to construct a triangulated mesh, reducing the number of dense point clouds in the vegetation area and improving the efficiency of 3D reconstruction.

[0088] The present invention adopts a deep learning method to identify and mark the vegetation areas in the image data. Therefore, after dense image matching, the land feature category (vegetation or non-vegetation) corresponding to each dense matching point can be obtained, and the vegetation land feature area in the dense point cloud can be resampled specifically.

[0089] Furthermore, the processing of the dense point cloud data using a thinning and resampling method to obtain a sampled dense point cloud specifically includes:

[0090] According to the three-dimensional space bounding box range of the dense point cloud and the preset spacing (for example, it can be set to 5 times the resolution of the ground object), the reconstructed scene is divided into a preset number of small space cubes C = {c1, c2, ..., c n}, count each cube c i The point set P(c i )={p1,p2,…,p m};

[0091] According to the cube c i The point set P(c i ) for each dense matching point (vegetation or non-vegetation), the category with the largest statistical number is used as the current cube c i The type of land features;

[0092] Select the cube c whose ground feature category is vegetation i , merge its point set P(c i ) in the dense matching points, the merged point coordinates are the set P(c i ) is the average value of all the point coordinates in ;

[0093] All cubes c whose ground objects are non-vegetation i The dense points in the and all merged vegetation points constitute the dense point cloud after sampling.

[0094] S40. Construct the sampled dense point cloud using a triangulated mesh method to obtain a three-dimensional surface mesh of the target scene, simplify the three-dimensional surface mesh using different strategies to obtain a simplified three-dimensional surface mesh model, and perform texture mapping based on the simplified three-dimensional surface mesh model to obtain a final three-dimensional model of the target scene; use the recognition result of the vegetation area to perform point cloud resampling and triangulated mesh simplification parameter setting.

[0095] After the resampling process of the above-mentioned dense point cloud, the sampled point cloud data can be used to construct a triangular mesh (for example, the classic Poisson meshing method can be used) to obtain a three-dimensional surface mesh of the target scene. In order to improve the subsequent texture mapping and the visualization efficiency of the generated three-dimensional scene, the common practice is to simplify the generated three-dimensional surface mesh, and then use the simplified model with less triangular face data to perform subsequent three-dimensional reconstruction steps. However, simplifying the three-dimensional surface mesh will inevitably cause the loss of geometric accuracy of the three-dimensional model. In this embodiment, in order to achieve a balance between scene reconstruction efficiency and geometric accuracy, different strategies are adopted in the process of triangular mesh simplification.

[0096] In this embodiment, a deep learning method is used to identify vegetation areas in oblique images and target scenes, and the identification results are applied to the multi-view image 3D reconstruction process to achieve the purpose of quickly reconstructing vegetation objects. Figure 3 As shown in FIG, according to the different proportions of vegetation areas in the target scene, the method proposed in the present invention can improve the reconstruction efficiency of the three-dimensional scene by 5% to 49%.

[0097] Furthermore, the simplification of the three-dimensional surface mesh using different strategies to obtain a simplified three-dimensional surface mesh model specifically includes:

[0098] For triangular meshes in non-vegetated areas, a smaller simplification effort can be used, while for less important vegetation areas, a larger simplification effort can be used. For example, if the classic QEM algorithm is used to simplify a 3D surface mesh, when calculating the edge collapse cost, if the vertex corresponding to the collapsed edge is a vegetation point, then the calculated edge collapse cost can be reduced to achieve the goal of simplifying the triangular face of the vegetation area as much as possible.

[0099] The embodiment of the present application further provides a vegetation model reconstruction system based on deep learning, the system comprising:

[0100] The image recognition module is used to identify vegetation objects in oblique images using deep learning methods, and obtain the vegetation object area in the reconstructed scene based on the image recognition results;

[0101] An image screening module is used to screen the recognition results of the vegetation feature area using an image screening method integrated with feature classification to obtain a screened image;

[0102] An image matching module is used to perform dense matching on the filtered images to obtain dense point cloud data of the reconstructed scene, and to process the dense point cloud data using a thinning and resampling method to obtain a sampled dense point cloud;

[0103] The model reconstruction module is used to construct the sampled dense point cloud using a triangular mesh method to obtain a three-dimensional surface mesh of the target scene, simplify the three-dimensional surface mesh using different strategies to obtain a simplified three-dimensional surface mesh model, and perform texture mapping based on the simplified three-dimensional surface mesh model to obtain the final three-dimensional model of the target scene.

[0104] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the above-described system and each module can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0105] The above-mentioned system can be implemented in the form of a computer program, and the computer program can be run on a computer device.

[0106] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A vegetation model reconstruction method based on deep learning, characterized in that: The method comprises: The vegetation features in the oblique image are identified using deep learning methods, and the identification results of the vegetation feature areas in the reconstructed scene are obtained based on the image recognition results; Performing image screening on the recognition result of the vegetation feature area using an image screening method integrated with feature classification to obtain a screened image; Performing dense matching on the filtered images to obtain dense point cloud data of the reconstructed scene, and processing the dense point cloud data using a thinning and resampling method to obtain a sampled dense point cloud; constructing the sampled dense point cloud using a triangular mesh method to obtain a three-dimensional surface mesh of the target scene, simplifying the three-dimensional surface mesh using different strategies to obtain a simplified three-dimensional surface mesh model, and performing texture mapping on the simplified three-dimensional surface mesh model to obtain a final three-dimensional model of the target scene; The method of identifying vegetation objects in the oblique image using a deep learning method and obtaining an identification result of the vegetation object area in the reconstructed scene based on the image recognition result includes: Using the UNet network in deep learning, pixel-level semantic segmentation is performed on the vegetation objects in the oblique image to determine the vegetation object area in the oblique image; Projecting each 3D connection point in the reconstructed scene onto all its observation images according to spatial triangulation to obtain observation image information corresponding to each 3D connection point; Determining the object category of each three-dimensional connection point based on the observed image information corresponding to each three-dimensional connection point, and obtaining an identification result of the vegetation object area in the reconstructed scene; The identification result of the vegetation feature area is subjected to image screening using an image screening method integrating feature classification to obtain a screened image, comprising: Obtaining a set of candidate visual observation images in the reconstructed scene using observation image information corresponding to each three-dimensional connection point; Using a greedy method to select the same number of images from the candidate visual observation image set as the screened images; The step of obtaining a set of candidate visual observation images in the reconstructed scene by utilizing the observation image information corresponding to each three-dimensional connection point comprises: According to the scope of the three-dimensional outer box of the reconstructed scene, the reconstructed scene is divided into a preset number of small space cubes according to a preset spacing; According to the three-dimensional spatial coordinates of the three-dimensional connection points, counting the point set of the three-dimensional connection points falling into each cube; Calculating a candidate visual observation image set for each cube based on observation image information corresponding to each three-dimensional connection point in the point set; The method of selecting the same number of images from the candidate visual observation image set using the greedy method as the filtered images includes: Using the observation image information corresponding to each three-dimensional connection point in the point set, a voting method is used to calculate the score of each visible image in the candidate visible observation image set of the cube; An upper limit on the number of images that can be selected from the cube is set according to a preset number of images in the candidate visual observation image set of the cube and a preset formula, wherein the preset formula is: Filtering, according to the score of each visible image in the candidate visible observation image set of the cube, cubes whose number of currently selectable images does not exceed the upper limit of the number of images; Counting the image with the largest score for each visible image in the cube whose number of filtered images does not exceed the upper limit of the number of images, taking it as the current best image, and setting the best image as the selected image; increasing the number of images selected by the cube including the best image in all the candidate visual observation image sets by 1, and deleting the best image from the candidate visual observation image set of the cube; Repeat the process: add 1 to the number of images selected by the cubes that contain the best image in all the candidate visual observation image sets, and delete the best image from the candidate visual observation image sets of the cubes, until the number of images selected by each cube reaches the maximum upper limit of the number of images set, and output all selected images as the filtered images.

2. The method according to claim 1, wherein The method of using the UNet network in deep learning to perform pixel-level semantic segmentation on the vegetation objects in the oblique image to determine the vegetation object area in the oblique image includes: Downsampling the image corresponding to the oblique image through an encoder in a UNet network body, and upsampling the image corresponding to the oblique image through a decoder in the UNet network body; Through the upsampling results, downsampling results and the output layer in the UNet network body, the probability that each pixel in the oblique image belongs to each ground object category is output, and the vegetation ground object area in the oblique image is determined based on the probability that each pixel belongs to each ground object category.

3. The method according to claim 1, wherein The step of determining the object category of each three-dimensional connection point based on the observed image information corresponding to each three-dimensional connection point and obtaining the recognition result of the vegetation object area in the reconstructed scene includes: Projecting each 3D tie point in the scene onto all its observation images according to the observation image information corresponding to each 3D tie point, and counting the ground feature categories to which pixels at the projection positions of the 3D tie point on different observation images belong; The ground object category with the largest number is determined as the ground object category to which the three-dimensional connection point belongs, and an identification result of the vegetation ground object area in the reconstructed scene is obtained.

4. The method according to claim 1, wherein The method of processing the dense point cloud data by using a thinning and resampling method to obtain a sampled dense point cloud includes: Dividing the reconstructed scene into a preset number of small spatial cubes according to the three-dimensional space bounding box range of the dense point cloud and at a preset spacing, and counting the point sets of dense matching points falling into each cube; According to the feature category of each dense matching point in the point set, the feature category with the largest number is used as the feature category of the corresponding cube; Select a cube with vegetation as the ground feature category, merge all dense matching points in the point set of the vegetation cube, and obtain all merged vegetation points in the reconstructed scene, where the coordinates of the merged vegetation points are the average of the coordinates of all points in the point set of each cube; The dense points in the cubes where all objects are classified as non-vegetation and all the merged vegetation points are taken as the dense point cloud after sampling.

5. The method according to claim 1, wherein The simplification of the three-dimensional surface mesh by different strategies to obtain a simplified three-dimensional surface mesh model includes: For the three-dimensional surface mesh of non-vegetated areas, the first level of simplification is used; A second level of simplification strength is applied to the three-dimensional surface mesh of the vegetation area to obtain a simplified model with less triangular face data, wherein the second level is greater than the first level.

6. A vegetation model reconstruction system based on deep learning, characterized in that: The vegetation object model reconstruction system based on deep learning is used to implement the vegetation object model reconstruction method according to claim 1, and the vegetation object model reconstruction system includes: The image recognition module is used to identify vegetation and ground objects in oblique images using a deep learning method, and obtain the recognition results of the vegetation and ground object areas in the reconstructed scene based on the image recognition results; An image screening module is used to screen the recognition results of the vegetation feature area using an image screening method integrated with feature classification to obtain a screened image; An image matching module is used to perform dense matching on the filtered images to obtain dense point cloud data of the reconstructed scene, and to process the dense point cloud data using a thinning and resampling method to obtain a sampled dense point cloud; The model reconstruction module is used to construct the sampled dense point cloud using a triangular mesh method to obtain a three-dimensional surface mesh of the target scene, simplify the three-dimensional surface mesh using different strategies to obtain a simplified three-dimensional surface mesh model, and perform texture mapping based on the simplified three-dimensional surface mesh model to obtain the final three-dimensional model of the target scene.

Images