Land use classification method, device and storage medium

Abstract

The present application discloses a land use classification method, device and storage medium, which relates to the field of data processing technology for land classification, in order to at least solve the problem of poor accuracy of land use classification results caused by insufficient data feature mining. The method includes: using multiple land use classification models, classifying the object categories according to the similarity between the feature information of the point cloud data and the feature information of the image data, and obtaining multiple first classification results; evaluating and analyzing the multiple first classification results, and taking the object category result with the highest evaluation as the second classification result; classifying the image into object categories based on object orientation, and obtaining an image including multiple contour areas of multiple target objects, and correcting the first target object category to which the pixel points in the contour area belong in the second classification result are the same as the pixel points in the contour area, based on the object category of the target object to which the pixel points in the contour area belong, to obtain a more accurate third classification result.

Description

Technical Field

[0001] The present application relates to the technical field of data processing for land classification, and in particular to a method, device and storage medium for classifying land use categories. Background Art

[0002] Human production activities and population growth affect changes in global land use, and the dynamic changes in land use affect and guide human production and living practices by changing the climate and ecological environment. Among them, land use feature categories are important indicators of urban morphology and the basis for extracting urban two-dimensional and three-dimensional morphological features. Therefore, conducting land use classification research is the basis for conducting research on urban two-dimensional and three-dimensional morphological features. However, in the current land use classification method, due to the single reference data source and poor application stability of the land classification algorithm, the accuracy of obtaining land use classification results is poor. Summary of the invention

[0003] The present invention provides a land use classification method, device and storage medium to at least solve the technical problem in the related art that the accuracy of the land use classification result is poor due to the single reference data source and the poor application stability of the land classification algorithm. The technical solution of the present invention is as follows:

[0004] According to a first aspect of an embodiment of the present invention, a method for classifying land use categories is provided, the method comprising: acquiring point cloud data from a laser radar scanner of an airborne LiDAR measurement system, and image data from an airborne digital camera; the resolution of the airborne digital camera is higher than a resolution threshold; using a plurality of different land use classification models, classifying the land object category to which each pixel belongs according to the feature similarity between the first multi-dimensional feature information of each pixel in the point cloud data and the second multi-dimensional feature information of each pixel in the image data being higher than a corresponding similarity threshold, and obtaining a plurality of first classification results; evaluating and analyzing the land object category to which each pixel in the plurality of first classification results belongs according to the frequency of belonging to the land object category to which each pixel in the plurality of first classification results belongs, and classifying the land object category to which each pixel with the highest evaluation result belongs. The first target object category of each pixel point is determined, and a second classification result including the first target object category to which each pixel point belongs is obtained; the image including each pixel point is segmented according to a preset segmentation scale, and the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information are used as the basis for determining the contour features of the object, and based on a preset merging condition, the segmented areas in the image are merged to obtain an image including multiple contour areas corresponding to multiple target objects; multiple pixels included in a contour area belong to one target object; based on the target object to which the pixel points in the contour area belong, the first target object category to which the pixel points in the second classification result that are the same as the pixel points in the contour area belong is corrected, and a third classification result including the second target object category of each pixel point is obtained.

[0005] The preset segmentation scales are associated with the object types to achieve multi-scale segmentation.

[0006] Specifically, the preset segmentation scale is determined based on the rate of change (ROC) of the local variance (LV) of the object homogeneity of different types of features.

[0007] The above-mentioned method of determining the object contour features using the elevation feature information in the first multidimensional feature information and the spectral feature information in the second multidimensional feature information specifically includes: determining the pixel points in the area corresponding to the preset segmentation size whose red band, green band and blue band are the same and whose difference between the relative elevation difference between the ground object and the ground is within a preset difference range as the pixel points of the same target object, and determining the pixel points in the pixel points of the area boundary corresponding to the preset segmentation size whose red band, green band and blue band are the same and whose difference between the relative elevation difference between the ground object and the ground is within a preset difference range as belonging to the same target object, so as to obtain the object contour features.

[0008] According to a second aspect of an embodiment of the present invention, a land use category classification device is provided, the device comprising: an acquisition unit, used to acquire point cloud data from a laser radar scanner of an airborne LiDAR measurement system, and image data from an airborne digital camera; the resolution of the airborne digital camera is higher than a resolution threshold; a first classification unit, used to adopt a plurality of different land use classification models, and classify the land object category to which each pixel point belongs according to the feature similarity between the first multi-dimensional feature information of each pixel point in the point cloud data and the second multi-dimensional feature information of each pixel point in the image data being higher than the corresponding similarity threshold, to obtain a plurality of first classification results; an evaluation unit, used to evaluate and analyze the land object category to which each pixel point in the plurality of first classification results belongs according to the frequency of belonging of the land object category to which each pixel point in the plurality of first classification results belongs, and classify the land object category to which each pixel point with the highest evaluation result belongs. The ground object category is determined as the first target ground object category of each pixel point, and a second classification result including the first target ground object category to which each pixel point belongs is obtained; a second classification unit is used to segment the image including each pixel point according to a preset segmentation scale, and use the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information as the basis for determining the object contour features, and based on a preset merging condition, merge the segmented areas in the image to obtain an image including multiple contour areas corresponding to multiple target objects; multiple pixels included in a contour area belong to one target object; a third classification unit is used to correct the first target ground object category to which the pixel points in the second classification result that are the same as the pixel points in the contour area belong based on the target object to which the pixel points in the contour area belong, and obtain a third classification result including the second target ground object category of each pixel point.

[0009] According to a third aspect of an embodiment of the present invention, there is provided a land use category classification system configured to execute the land use category classification method of the first aspect and any possible implementation thereof.

[0010] According to a fourth aspect of an embodiment of the present invention, a computer device is provided, comprising: a processor and a memory for storing processor executable instructions; wherein the processor is configured to execute executable instructions to implement a land use category classification method such as the first aspect and any possible implementation thereof.

[0011] According to a fifth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium having instructions stored thereon. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer device is enabled to execute a land use category classification method such as the first aspect and any possible implementation thereof.

[0012] According to the sixth aspect of the embodiment of the present application, a computer program product is provided, which includes computer instructions. When the computer instructions are executed on a computer device, the computer device executes the land use category classification method of the first aspect and any possible implementation thereof.

[0013] The technical solution provided by the embodiment of the present invention brings at least the following beneficial effects: based on the feature similarity between the first multi-dimensional feature information of each pixel in the point cloud data and the second multi-dimensional feature information of each pixel in the image data, the feature classification of the object category is determined, so that each first classification result is obtained based on the data of multiple data sources of different dimensions, thereby ensuring the accuracy of each first classification result. And each first classification result is evaluated, so that the object category classification result with the best evaluation result is used as the second classification result, so as to reduce the influence of the instability in the application process of each single classification algorithm on the first classification result, thereby ensuring that the second classification result is more accurate. Further, in order to avoid the "salt and pepper" phenomenon in the classification result due to the excessive consideration of the data dimension, that is, the distribution of the two-dimensional and three-dimensional target objects is relatively fragmented, based on the two-dimensional information of the elevation feature information and the spectral feature information, an object-oriented classification method is adopted to further classify the object category of the image, so as to determine the target object to which the multiple pixels included in the contour area belong, and based on the target object classification result, the "salt and pepper" phenomenon problem that may appear in the second classification result is corrected, and finally a more accurate third classification result is obtained. Therefore, through the land use category classification method of the present application, full use is made of the multi-dimensional feature advantages embodied by multiple data sources to improve the granularity of classification, and multiple classification algorithms are evaluated to avoid the impact of poor application stability of land classification algorithms on the accuracy of classification results. At the same time, an object-oriented classification method is used to correct possible "salt and pepper" problems, thereby improving the accuracy of land use category classification results.

[0014] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application, and do not constitute an improper limitation on the present application.

[0016] Figure 1 is a flow chart showing a land use classification method according to an exemplary embodiment;

[0017] Figure 2is a diagrammatic diagram showing a result of data fusion of different feature dimensions according to an exemplary embodiment;

[0018] Figure 3 is a schematic block diagram of a land use classification device according to an exemplary embodiment;

[0019] Figure 4 The diagram is a schematic diagram of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0020] In order to enable ordinary persons in the art to better understand the technical solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0022] Before introducing in detail the land use category classification method provided in the embodiment of the present application, a brief introduction to the application scenarios involved in the embodiment of the present application is first given.

[0023] Human production activities and population growth affect changes in global land use, and the dynamic changes in land use affect and guide human production and living practices by changing the climate and ecological environment. Changes in LST (Land Surface Temperature) are the product of the continuous evolution of land use, and the spatial distribution of land use types is an important indicator of urban morphology and the basis for extracting urban two-dimensional morphological features. Therefore, conducting land use classification research is the basis for studying the impact of urban two-dimensional morphological features on LST. However, in the current land use classification method, due to the single reference data source and poor stability of land classification algorithm application, the accuracy of obtaining land use classification results is poor.

[0024] Based on this, the following research was conducted. The emergence of satellite remote sensing technology has enabled large-scale terrain observation of the earth's surface. Remote sensing sensors can sense and record the spectral reflectance of different surface cover features from visible light to infrared bands and from medium to extremely high spatial resolution, which is conducive to better interpretation and processing of images. Remote sensing image information extraction is the basis of many remote sensing application studies. However, how to fully explore and utilize useful remote sensing information to identify and classify complex land object types is an urgent problem to be solved in remote sensing research. At present, high spatial resolution images, multi / hyperspectral images, and lidar have been widely used in land use classification research. However, with the continuous development of cities, land object types are becoming more and more diverse and complex. A single remote sensing data source can no longer meet the extraction of complex urban land use object categories. On the other hand, multi- / hyperspectral remote sensing images can obtain more or continuous spectral information of objects, but the spatial resolution of most images is not high, which makes it difficult to meet the requirements of fine land use classification; laser scanners can obtain high-precision three-dimensional spatial information of the surface without being affected by lighting conditions, but lidar data lacks sufficient spectral information and is difficult to identify objects with similar height characteristics; airborne aerial images can obtain high-resolution images in the visible light band, which can achieve synchronous collection and acquisition with LiDAR point cloud data. Therefore, the fusion of airborne LiDAR and image data can not only achieve complementary advantages, but also eliminate the problem of asynchronous data acquisition time, making high-resolution urban land use classification research possible.

[0025] The fusion of multi-source remote sensing data can effectively achieve complementary advantages, but the mining of different remote sensing data information is another important research content. The study found that direct features of LiDAR point cloud such as elevation and intensity information can effectively improve the classification accuracy of objects. By fusing multispectral data and DSM, the classification accuracy of objects can be improved by 5%-6%; with the development of point cloud filtering algorithms, the extraction accuracy of normalized digital surface models has been greatly improved, and nDSM (i.e., the relative elevation difference between objects and the ground) can eliminate the influence of terrain and greatly improve the accuracy of object classification, especially in distinguishing low vegetation from high vegetation; LiDAR intensity information can effectively distinguish objects with different surface materials. The intensity model is extracted by interpolating the intensity information and applied to land use classification, which improves the classification accuracy of objects. A large number of features derived from the height information of LiDAR point clouds are widely used in the classification of land objects, such as the skewness model, kurtosis model, mean, variance and standard deviation based on elevation. Although these features are helpful to improve the classification accuracy of land objects, the degree of improvement is not as good as the normalized digital surface model and intensity model. In addition, the texture information based on elevation can effectively assist the normalized digital surface model and image to extract land objects in the classification of land objects. Most of the above features are based on the change of height information for feature description, and the local geometric change of elevation can also effectively express the difference between land objects. The surface roughness model is constructed by calculating the undulation of the surface of the land object in the unit grid, and the classification accuracy of land use is effectively improved. Based on LiDAR point cloud data, flatness, normal change and texture features based on the normalized digital surface model are extracted at the grid scale, and applied to the automatic extraction of buildings, and good results are obtained. Therefore, the above research findings show that the mining and effective combination of LiDAR point cloud information can greatly improve the classification accuracy of land objects.

[0026] In addition to spectral information, remote sensing images can also extract spatial distribution characteristics from the perspective of image morphology. Using WorldView-2 images, the spectral-spatial characteristics were calculated based on the three-dimensional wavelet transform method, and the multi-scale urban complexity index was applied to the classification of objects. The morphological building index and building shadow index were extracted based on spatial morphology for building extraction; the land use classification was studied by fusing texture features, morphological features, normalized vegetation index and normalized water index, and the DSM generated based on the Resource-3 stereo image pair. On this basis, multi-scale urban change monitoring was carried out, and good results were obtained. In the above studies, we found that most morphological features were extracted based on optical images, and were not fused with other data sources such as LiDAR data for object classification research.

[0027] In view of the above research findings, the present application provides a land use classification method, which determines the classification of land objects based on the feature similarity between the first multi-dimensional feature information of each pixel in the point cloud data and the second multi-dimensional feature information of each pixel in the image data, so that each first classification result is obtained based on data from multiple data sources of different dimensions, thereby ensuring the accuracy of each first classification result. And each first classification result is evaluated to use the land object classification result with the best evaluation result as the second classification result, so as to reduce the impact of the instability in the application process of each single classification algorithm on the first classification result, thereby ensuring that the second classification result is more accurate. Further, in order to avoid the "pepper and salt" phenomenon in the classification result due to the excessive consideration of the data dimension, that is, the distribution of the two-dimensional and three-dimensional target objects is relatively fragmented, based on the two-dimensional information of the elevation feature information and the spectral feature information, an object-oriented classification method is adopted to further classify the land object category of the image, so as to determine the target object to which the multiple pixels included in the contour area belong, and based on the target object classification result, the "pepper and salt" phenomenon problem that may appear in the second classification result is corrected, and finally a more accurate third classification result is obtained.

[0028] The land use classification method provided in the embodiment of the present application can be applied to a computer device having a land use classification system or for land use classification. For ease of understanding, the land use classification method provided in the present application is specifically introduced below in conjunction with the accompanying drawings.

[0029] Figure 1 is a flow chart of a land use classification method according to an exemplary embodiment. Figure 1 As shown, the land use category classification method includes the following steps.

[0030] S11, obtains point cloud data from the laser radar scanner of the airborne LiDAR measurement system, and image data from the airborne digital camera.

[0031] In order to ensure the accuracy of the image data, the resolution of the onboard digital camera is set to be higher than the resolution threshold.

[0032] It is understandable that the above-mentioned point cloud data and image data are image data for the land area to be classified.

[0033] In some embodiments, the point cloud data is airborne LiDAR point cloud data, and the image data may also be referred to as airborne image data. The point cloud data includes ground points and non-ground points, and is discrete point cloud data.

[0034] In some embodiments, the airborne LiDAR measurement system can synchronously acquire LiDAR point cloud data and aerial image data at the same time. The airborne laser scanning system is integrated with hardware such as RIEGL VUX-1LR laser radar scanner, PHASE ONE IXU1000-R high-resolution digital camera, and POS composed of IMU and differential GPS. The entire system is mounted on a flight platform or aircraft (such as a manned aircraft or drone) for data collection. During the data collection process, the laser radar scanner is used to obtain the X, Y, Z three-dimensional coordinates and intensity information of the target object in real time, and the digital camera is used to obtain high-resolution image data of the target object in the visible light band. The POS system performs combined navigation through IMU inertial navigation information and differential GPS positioning information to obtain the carrier's motion posture and position information (i.e., POS trajectory information) for subsequent point cloud fusion and solution and image correction.

[0035] The point cloud data used above has undergone preliminary preprocessing, including the acquisition of original data, calculation of POS trajectory, and fusion of POS and laser point cloud data to generate point cloud data in the geographic coordinate system, which will not be repeated here.

[0036] In an embodiment of the present application, the orthorectification of the airborne image data is achieved by using model key points extracted from point cloud ground points based on point cloud data as ground control points. Therefore, the geographic coordinates of various types of feature information extracted based on point cloud data and image data are corresponding and unified, so as to facilitate the unified definition of the coordinate systems of feature information in each dimension.

[0037] Optionally, the minimum elevation of the point cloud data is 338m and the maximum elevation is 519.19m. The average point cloud density of the entire study area is 20 pts / m2, the horizontal resolution is 0.17m, the vertical resolution is 0.2m, and the maximum height difference is 181.19m. The image has a width of 11608pixel×8708pixel and a resolution of 3.33 mm, including the three spectral bands of red, green, and blue in the visible light band.

[0038] S12, using multiple different land use classification models, classify the land object category to which each pixel point belongs according to the feature similarity between the first multi-dimensional feature information of each pixel point in the point cloud data and the second multi-dimensional feature information of each pixel point in the image data being higher than the corresponding similarity threshold, and obtain multiple first classification results.

[0039] The above-mentioned point cloud data and image data are respectively calibrated point cloud data and image data.

[0040] Specifically, the attitude information and position information of the aircraft are obtained from the positioning device of the airborne LiDAR measurement system; the positioning device includes an IMU (Inertial Measurement Unit) inertial measurement unit and a differential GPS (Differential Global Positioning System) differential global positioning system; the point cloud data and image data are corrected respectively based on the attitude information and position information of the aircraft to obtain corrected point cloud data and image data; the elevation of the corrected point cloud data is within a preset elevation range, and the average point cloud density of the point cloud data is within a preset density range; the horizontal resolution of the point cloud data is within a first preset resolution; the vertical resolution of the point cloud data is within a second preset resolution; the resolution of the image data is within a third preset resolution; and the width of the image data is within a preset width.

[0041] In one embodiment, indirect feature information and direct feature information of each pixel are extracted from the point cloud data as the first multi-dimensional feature information, and spectral feature information, visible light vegetation index and image space feature information of each pixel are extracted from the image data as the second multi-dimensional feature information.

[0042] Among them, direct feature information includes elevation feature information and intensity feature information; indirect feature information includes spatial geometric feature information of point cloud and texture feature information used to describe the correlation between the grayscale of two points within a preset distance and in a preset direction; spatial geometric feature information includes the undulation of the object on the surface, the flatness of the object surface and the discreteness of the point cloud; image spatial feature information includes: morphological building index, morphological shadow index and urban complexity.

[0043] Exemplarily, the spectral characteristic information includes information of three spectral bands of red, green and blue in the visible light band.

[0044] Based on this, multiple different land use classification models are adopted. According to the feature similarity between the indirect feature information, direct feature information, spectral feature information, visible light vegetation index and image space feature information of each pixel point is higher than the corresponding similarity threshold, the land object category to which each pixel point belongs is classified to obtain multiple first classification results.

[0045] In some embodiments, the above-mentioned multiple different land use classification models are different machine learning models; the multiple different land use classification models can be K-Nearest Neighbor (KNN) classifier algorithm model, Random Forest (RF) classifier algorithm model and extreme gradient boosting (eXtreme GradientBoosting, XBGoost) classifier algorithm model.

[0046] Among them, the KNN classifier algorithm model is an instance-based learning method. Its core idea is to determine the category of the sample to be classified by calculating the distance between the sample to be classified and the sample in the training set, and selecting the categories of the K samples closest to vote. In land use classification, the KNN classifier algorithm model can compare the characteristics of each pixel with the characteristics of its neighboring pixels to perform pixel-level classification.

[0047] The RF classifier algorithm model is an integrated learning algorithm that performs classification by constructing multiple decision trees and combining their results. In land use classification, the RF classifier algorithm model can use a large number of decision tree models to classify land objects, and the category of each pixel point is obtained by voting from multiple decision trees.

[0048] The XGBoost classifier algorithm model is a gradient boosting tree algorithm that iteratively trains multiple weak classifiers and combines them into a strong classifier. In land use classification, the XGBoost classifier algorithm model can classify land objects by constructing a gradient boosting tree model, and it performs well in processing large-scale data and high-dimensional features.

[0049] The XGBoost classifier algorithm model priority is greater than the RF classifier algorithm model priority; the RF classifier algorithm model priority is greater than the KNN classifier algorithm model priority.

[0050] The training process of the above-mentioned multiple different land use classification models is as follows.

[0051] First, construct multiple different initial models that represent the association between each feature information of point cloud data and image data and the ground object category to which each pixel point belongs. The multiple initial models may include a KNN classifier algorithm model, an RF classifier algorithm model, and an XGBoost classifier algorithm model.

[0052] Secondly, the historical indirect feature information and historical direct feature information of each historical pixel point are extracted from multiple sets of historical point cloud data, and the historical spectral feature information, historical visible light vegetation index and historical image space feature information of each historical pixel point are extracted from multiple sets of historical image data corresponding to the multiple sets of historical point cloud data, and multiple sets of historical images corresponding to the multiple sets of historical point cloud data are obtained, in which the feature category to which each historical pixel point in the historical image belongs has been marked.

[0053] Third, according to preset proportion rules, multiple sets of historical point cloud data, multiple sets of historical image data and multiple sets of historical images are selected into training sample data sets and verification sample data sets.

[0054] In order to avoid the impact of imbalance in sample quantity and proportion on classifier training, the preset proportion rule refers to selecting the number of samples for each land feature category based on the proportion of each land feature category in the historical image area.

[0055] Fourthly, multiple initial models are trained separately based on the training sample data set, and the validation sample data set is used to validate each trained initial model, so that each initial model with the best validation result is used as multiple different land use classification models.

[0056] The above-mentioned land use classification model may be a classification model composed of various classification algorithms that can realize pixel-level land use classification. The present application does not make any specific limitation on the land use classification model.

[0057] In some embodiments, the above-mentioned land use classification model is also referred to as a classification scheme.

[0058] In some machine learning implementations, the selection of classification samples is an extremely critical step, which directly affects the establishment of the model and the quality of the classification results. The sample data set includes a training data set and a test data set. The training data set is used to produce a learner, and the test data set is used to test the learner's ability to discriminate new samples. The test error on the test set is used as an approximation of the generalization error to select the learner. It is usually assumed that the training set and the test set are sampled independently and identically from the sample set, and the samples in the training set and the test set are mutually exclusive, that is, the data do not overlap.

[0059] In some implementations, based on the spectral response of the objects in the aerial images and field observations, the object types in the study area (i.e., the land area to be classified) are divided into seven land use types: building land, bare land, cultivated land, grassland, road, forest land, and water. In order to avoid the impact of the imbalance of sample quantity and proportion on classifier training, a random sampling method is used to establish a sample data set covering the entire study area and all categories, and the number of samples for each category is selected according to the proportion of the category in the entire study area.

[0060] The selection of sample data sets mainly includes the following steps: first, randomly create sample points covering the entire study area, and these samples do not contain any characteristic attribute information; second, combine high-resolution Google Earth images with similar imaging time and aerial true color images to classify the sample points. Then, the random sampling method is used again to divide the sample points into 80% training data sets and 20% test data sets (there is no overlap between the training data sets and the test data sets). Finally, the generated synthetic images extract the characteristic information (band information) contained in the synthetic images of each classification scheme into the corresponding training sample points and test sample points as their attribute values.

[0061] Exemplarily, first, the sample data set is randomly divided into 80% training sample data set and 20% validation sample data set. Secondly, the three classifier algorithms of KNN, RF and XGBoost are used to learn and train the training sample data set respectively, and a training model is constructed. Next, the accuracy of the calculation model is calculated using the validation sample data set, and the model parameters are optimized by analyzing the classification results. At this time, there is a cyclic iterative process until the optimal training model and the best classification accuracy are obtained; finally, the final classification results of the above three classifier algorithms are evaluated and analyzed, and the impact of each input feature and classification scheme on the land use classification results is evaluated and analyzed by comparing and analyzing the evaluation indicators such as OA, UA, PA, CA and kappa coefficient of each classification scheme.

[0062] Optionally, when the Kappa coefficient is between 0.8 and 1.0, it represents almost perfect consistency (Almost perfect); when the Kappa coefficient is between 0.6 and 0.8, it represents high consistency (Substantial); when the Kappa coefficient is between 0.4 and 0.6, it represents moderate consistency (Moderate); when the Kappa coefficient is between 0.2 and 0.4, it represents fair consistency (Fair); when the Kappa coefficient is between 0 and 0.2, it represents very low consistency (Slight).

[0063] In this step, with the continuous advancement of urbanization, the types of land objects have become more and more diverse, making it difficult to achieve high-precision classification and extraction of a single land object category. The use of high-resolution airborne aerial images can obtain clear land object coverage, but the types of land objects with similar spectra cannot be further distinguished, such as roads, buildings and other surfaces with the same material. The introduction of airborne LiDAR point cloud data (such as elevation information) can make up for this defect. It is not only conducive to the classification of land objects with different elevation distributions or surface materials, but also can be obtained synchronously with image data to avoid data inconsistency caused by different data collection times. The fusion of airborne LiDAR data (i.e., point cloud data) and image data can improve the accuracy of land use classification with high resolution.

[0064] S13, based on the frequency of the ground object category to which each pixel point belongs in the multiple first classification results, evaluate and analyze the ground object category to which each pixel point in the multiple first classification results belongs, and determine the ground object category to which each pixel point with the highest evaluation result belongs as the first target ground object category of each pixel point, and obtain a second classification result including each first target ground object category to which each pixel point belongs.

[0065] In one embodiment, for any pixel point among the pixel points, the ground object category to which the pixel point belongs most frequently in the multiple first classification results is determined as the first target ground object category of the pixel point, so as to obtain a second classification result including the first target ground object categories to which the pixel point belongs.

[0066] As another implementation method, for any pixel point among each pixel point, if among the multiple feature categories to which any pixel point in multiple first classification results belongs, there is a same feature category with the largest number of belongings and the same feature category is unique, then the same feature category with the largest number of belongings is determined as the feature category to which any pixel point with the highest evaluation result belongs; if among the multiple feature categories to which any pixel point in multiple first classification results belongs, there is a same feature category with the largest number of belongings and the same feature category is greater than or equal to 2, then the same feature category with the highest priority in the first classification results among the multiple same feature categories with the largest number of belongings is determined as the feature category to which any pixel point with the highest evaluation result belongs.

[0067] Among them, the priorities of multiple first classification results are positively correlated with the priorities of multiple corresponding different land use classification models; the priorities of multiple different land use classification models are determined based on the adaptability of the first multidimensional feature information and the second multidimensional feature information to the land use classification model or the influencing factors of the land use classification model.

[0068] The specific process of determining the fitness or influencing factor is as follows: the single-dimensional feature information of the single data source in the first multi-dimensional feature information and the second multi-dimensional feature information is separately input into different land use classification models, and multiple single-dimensional feature information of the multi-data sources in the first multi-dimensional feature information and the second multi-dimensional feature information are combined and respectively input into different land use classification models, and the classification results under the single-dimensional feature information and the classification results under the multi-dimensional feature information are obtained correspondingly. Based on the classification results under the above single-dimensional feature information and the classification results under the multi-dimensional feature information, the influence of each dimensional feature information in the first multi-dimensional feature information and the second multi-dimensional feature information on each land classification model is analyzed, and each fitness of each land use classification model or each influencing factor of each land use classification model according to the first multi-dimensional feature information and the second multi-dimensional feature information is obtained, so as to determine the priority of each land use classification model according to each fitness of each land use classification model or each influencing factor of each land use classification model, thereby obtaining the priority of each first classification result output by each land use classification model.

[0069] For example, Figure 2As shown in the figure, the elevation features, intensity features, spatial geometric features and texture features extracted from airborne LiDAR data (or airborne LiDAR point cloud data) and the spectral features, visible light vegetation index and image space features extracted from airborne image data are combined to explore the influence of different input features of different data sources on the land use classification results. Specifically, three groups of twelve different land use classification schemes were designed based on single data source and multiple data sources. First, three land use classification schemes (i.e., Schemes 1-3) were designed using only airborne image data as the data source to compare and analyze the contribution or influence of spectral characteristics, spatial characteristics and visible light vegetation index on the accuracy of land use classification. Second, four land use classification schemes (i.e., Schemes 4-7) were designed using only airborne LiDAR data as the data source to compare and analyze the contribution or influence of elevation, intensity, spatial geometric characteristics and texture characteristics on land use classification. Finally, five land use classification schemes (i.e., Schemes 8-12) were designed by integrating airborne LiDAR and image data as data sources to explore the impact of the fusion of direct features and derived features (i.e., indirect features) extracted from different data sources on improving the accuracy of land use classification, and at the same time, a comparative analysis was conducted with the first two groups of land use classification using single data sources.

[0070] Correspondingly, in Figure 2In Scheme 1, only spectral feature information is input, and a synthetic image with a corresponding number of bands (i.e., number of features) of 3 is generated. In Scheme 2, only spectral feature information and spectral spatial features are input, and a synthetic image with a corresponding number of bands (i.e., number of features) of 6 is generated. In Scheme 3, only spectral feature information, image spatial features, and visible light vegetation index are input, and a synthetic image with a corresponding number of bands (i.e., number of features) of 14 is generated. In Scheme 4, only elevation feature information is input, and a synthetic image with a corresponding number of bands (i.e., number of features) of 1 is generated. In Scheme 5, only elevation feature information and intensity feature information are input, and a synthetic image with a corresponding number of bands (i.e., number of features) of 2 is generated. In Scheme 6, only elevation feature information, intensity feature information, and spatial geometric feature information are input, and a synthetic image with a corresponding number of bands (i.e., number of features) of 5 is generated. In Scheme 7, only elevation feature information, intensity feature information, spatial geometric feature information, and texture feature information are input, and a synthetic image with a corresponding number of bands (i.e., number of features) of 13 is generated. In scheme 8, only spectral feature information and elevation feature information are input, and a synthetic image with a corresponding band number (i.e., feature number) of 4 is generated. In scheme 9, only spectral feature information, elevation feature information, and intensity feature information are input, and a synthetic image with a corresponding band number (i.e., feature number) of 5 is generated. In scheme 10, only spectral feature information, elevation feature information, intensity feature information, and spatial geometric feature information are input, and a synthetic image with a corresponding band number (i.e., feature number) of 8 is generated. In scheme 11, only spectral feature information, elevation feature information, intensity feature information, spatial geometric feature information, and texture feature information are input, and a synthetic image with a corresponding band number (i.e., feature number) of 16 is generated. In scheme 12, only spectral feature information, building space feature information, visible light vegetation index, elevation feature information, intensity feature information, spatial geometric feature information, and texture feature information are input, and a synthetic image with a corresponding band number (i.e., feature number) of 27 is generated.

[0071] S14, according to a preset segmentation scale, the image including each pixel point is segmented, and the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information are used as the basis for determining the contour features of the object, and based on a preset merging condition, the segmented areas in the image are merged to obtain an image including multiple contour areas corresponding to multiple target objects.

[0072] It can be understood that according to the preset segmentation scale and the preset merging conditions, and taking the elevation feature information in the first multidimensional feature information and the spectral feature information in the second multidimensional feature information as the basis for determining the contour features of the object, the image including each pixel point is classified into ground object categories on an object-oriented basis to obtain an image including multiple contour areas of multiple target objects.

[0073] Among them, a plurality of pixel points included in a contour area belong to a target object.

[0074] It can be understood that each segmentation and merging process mentioned above is executed synchronously, that is, the image including each pixel point is segmented according to a preset segmentation scale. After one segmentation, the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information are used as the basis for determining the contour features of the object. Based on the preset merging conditions, the segmented areas in the image are merged until the last segmentation is completed and the last merging of the segmented areas is completed, so as to obtain an image including multiple contour areas corresponding to multiple target objects.

[0075] The preset segmentation scale is associated with the object type to achieve multi-scale segmentation. It is understandable that different object types have different optimal segmentation scales. By selecting different segmentation scales for different types of objects, the segmentation layer of the object can be better constructed, effectively avoiding the phenomenon of over-segmentation or under-segmentation of the object.

[0076] In some methods for determining a preset segmentation scale, a segmentation size interval is determined, and the image is segmented at multiple scales to generate multiple segmentation layers with segmentation sizes ranging from small to large; based on the homogeneity index and local Moran index of the segmented object, an evaluation index of each segmentation object in each segmentation layer is calculated; based on the evaluation index, the optimal segmentation object in each segmentation layer is selected, and the image is segmented again using the attribute value of the optimal segmentation object as the vector constraint boundary to generate a segmentation layer composed of the optimal segmentation objects, and the size of the segmentation layer composed of the optimal segmentation objects is used as the preset segmentation scale.

[0077] The above-mentioned preset merging conditions include one or more of the following: minimizing object heterogeneity or minimizing image smoothness heterogeneity, minimizing image compactness heterogeneity, minimizing image shape heterogeneity and minimizing average heterogeneity of all objects; the preset segmentation scale is determined according to the object type of the segmented object.

[0078] It can be understood that multi-scale segmentation is a bottom-up region merging algorithm that starts with arbitrary pixels (i.e., pixel points). It is more commonly used for image segmentation, and its segmented objects are usually well aligned with natural boundaries and evenly distributed.

[0079] In some embodiments, the multi-scale segmentation algorithm uses minimization of object heterogeneity as a criterion to merge any pixel in the image with its adjacent objects. A segmentation scale parameter is set here. When the heterogeneity between adjacent objects is greater than the square of the parameter, the merging is stopped, otherwise the merging of adjacent objects continues.

[0080] Spectrum and shape factor are two important parameters that affect the heterogeneity of images. The shape factor includes smoothness heterogeneity and compactness heterogeneity. Only when the smoothness heterogeneity, compactness heterogeneity and shape heterogeneity of the image are minimized can the average heterogeneity of all objects be minimized.

[0081] Given that the main parameters affecting multi-scale segmentation include the segmentation scale and the heterogeneity criterion of the preset merging condition points, the segmentation scale controls the acceptable heterogeneity of the segmentation results under the action of the heterogeneity criterion, and the heterogeneity criterion can be adjusted by the shape factor and the color factor.

[0082] Taking the total heterogeneity as the weighted sum of spectral and shape heterogeneity as an example, the process of determining the heterogeneity standard characterized by the preset merging condition is explained as follows based on the following formula.

[0083] (Formula 1).

[0084] in, is the spectral heterogeneity weight value; is the shape heterogeneity weight value; spectral heterogeneity is and shape heterogeneity is ;in, .

[0085] (Formula 2).

[0086] (Formula 3).

[0087] in, is the object standard deviation composed of the grayscale values ​​of the cth band; is the weight of the standard deviation at different scales; is the smoothness weight; is the compactness weight value, and the smoothness heterogeneity is and compactness heterogeneity is ; .

[0088] Smoothness heterogeneity and compactness heterogeneity can be determined by the following formulas.

[0089] (Formula 4).

[0090] (Formula 5).

[0091] In the above formula, L is the side length of the segmented object; n is the number of pixels in the object area; and b is the shortest side length of the circumscribed rectangle of the object in the horizontal direction.

[0092] After the image objects are merged, their spectral heterogeneity is , shape heterogeneity is , can be determined by the following formula.

[0093] (Formula 6).

[0094] (Formula 7).

[0095] Among them, the compactness heterogeneity of the merged region is and the smoothness heterogeneity is .

[0096] (Formula 8).

[0097] (Formula 9).

[0098] In the above formula, Indicates the side length of the merged object area; and is the side length of the two objects before merging; is the number of pixels in the area after the object is merged; and is the number of pixels in the two object areas before merging; is the shortest side length of the horizontal circumscribed rectangle of the merged object area; and It is the shortest side length of the horizontal circumscribed rectangle of the two object areas before merging.

[0099] In the above steps, based on the three main parameters that affect the multi-scale segmentation results, i.e., the multiple contour areas corresponding to the multiple target objects, namely, the segmentation scale, the image layer weight, and the homogeneity factor, the segmentation scale is set as the most important parameter affecting the multi-scale segmentation algorithm; the image layer weight is set as the weight of each band in the input image; the homogeneity factor is a weight jointly determined by the shape and color factors, where the shape parameter is the weighted percentage of the shape uniformity relative to the uniformity of the color (i.e., the spectral value), and the compactness parameter is a sub-parameter of the shape factor, which is used to optimize the compactness and smoothness of the image segmentation object.

[0100] Specifically, a weight setting process is as follows.

[0101] First, determine the weight of the input band. Based on the influence factors of the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information on the adaptability of the land use classification model or the land use classification results, determine the weight of the input band to determine the image spectral heterogeneity in image segmentation. Among them, different input bands contribute differently to the classification of each land object. In the process of image segmentation, the weighted bands include RGB of spectral feature information and nDSM of elevation feature information. Optionally, according to the importance analysis of the bands and after multiple experimental parameter adjustments, the weight values ​​of each band in the image to be segmented are adjusted so that the weights of the red, green, and blue bands are all set to 1, and the weight of nDSM is set to 1.5.

[0102] Second, determine the homogeneity factor. The homogeneity factor settings of the objects in the image include shape parameters and compactness parameters. The shape parameter corresponds to the color, and the sum of the weights of the two is 1. For example, if the shape parameter is set to 0.7, the color parameter is automatically 0.3. The compactness parameter corresponds to the smoothness, and the sum of the weights of the two is also 1. For example, if the compactness parameter is set to 0.6, the smoothness parameter is automatically 0.4. The quality of the image segmentation results is closely related to the shape factor and the color factor. The shape factor affects the differences in the geometric features of different objects, while the color factor affects the differences in the spectral features of the objects.

[0103] In some embodiments, spectral features are important image parameters. Therefore, when setting segmentation parameters, spectral feature information should be used as the main segmentation basis as much as possible while satisfying the shape factor. For linear features such as roads and rivers, the smoothness weight can be increased and the compactness weight can be appropriately reduced; while for regularly distributed features such as buildings and cultivated land, the compactness parameter can be appropriately set higher.

[0104] Optionally, the shape parameter is set to 0.5, the corresponding color factor weight is 0.5, and the compactness parameter is set to 0.5, the corresponding smoothness factor weight is 0.5.

[0105] Third, determine the preset segmentation scale. The smaller the segmentation scale, the more objects in the segmented image, the lower the computational efficiency, and the higher the fragmentation of the objects in the image, which is prone to over-segmentation. When the segmentation scale is large, the fewer objects are and the larger the area is, the smaller objects are likely to be merged, resulting in under-segmentation. If the segmentation scale is too large or too small, it will seriously affect the quality of object segmentation. Therefore, choosing an appropriate segmentation scale is crucial for object segmentation.

[0106] Specifically, the scale parameter with the best object segmentation effect is determined by calculating the rate of change (ROC) of the local variance (LV) of the homogeneity of the objects in the image under different segmentation scale parameters. The specific determination process is as follows.

[0107] (Formula 10).

[0108] in, is the local variance of the current target layer; For The local variance of the next target layer based on the baseline.

[0109] When the rate of change of LV is the largest, that is, when a peak value appears, the segmentation scale corresponding to this point is the optimal segmentation scale. When there is not only one optimal segmentation scale, this is because multiple optimal segmentation scales are obtained for different objects in the image. The above multiple optimal segmentation parameters can be used as alternatives, combined with other parameter settings, and the multi-scale segmentation algorithm can be used to achieve the segmentation of objects in the image.

[0110] In some embodiments, after the input band weight and homogeneity factor parameters are determined, the optimal segmentation scale (i.e., the preset segmentation scale) is determined. Exemplarily, the resolution of the input image data in the study area is 1m. When the initial segmentation scale is less than 10, the segmentation merging calculation time will be greatly increased. Therefore, the initial segmentation scale is determined to be 10, the segmentation step is 10, and the number of iterations is 49 times, and finally the change curve of the local change LV and the change rate value ROC of homogeneity is obtained. Based on the change of observed ROC, it is obtained that with the increase of the segmentation scale, LV shows a trend of gradual increase, and finally tends to be gentle, while ROC shows a sharp decline, and then presents a sawtooth fluctuation. According to the change of ROC and removing the larger scale value, the optimal segmentation scale of the alternative is finally determined to be 90, 150, 320 and 410. Through comparative analysis, when the segmentation scale is 90, the details of the target object can be better retained. Therefore, the segmentation scale of 90 is used as the optimal segmentation scale.

[0111] In one embodiment, the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information are used to determine the contour features of the object, specifically including: using the airborne RGB image data as the reference image for segmentation, that is, the image, taking into account the importance of the elevation feature information to the classification of the ground object, using nDSM (that is, the relative elevation difference between the ground object and the ground) as auxiliary data to jointly participate in the object segmentation, and determining the contour of the object based on the consistency of the four bands contained in the image, namely, the red band, the green band, the blue band and the nDSM. Specifically, the red band, the green band, and the blue band in the area corresponding to the preset segmentation size are the same, and the difference between the nDSMs is within the preset difference range. The pixel points are determined as the same target object, and the red band, the green band, and the blue band in the pixel points adjacent to the area boundary corresponding to the preset segmentation size are the same, and the difference between the nDSMs is within the preset difference range. The pixel points are determined as belonging to the same target object.

[0112] S15, based on the target object to which the pixel points in the contour area belong, correct the first target object category to which the pixel points identical to the pixel points in the contour area in the second classification result belong, to obtain a third classification result of the second target object category including each pixel point.

[0113] As a correction method, when the number of pixel points of the first target object category consistent with the object category of the target object included in any contour area is greater than or equal to the number corresponding to a preset proportion of the number of all pixel points in any contour area, for any pixel point in each pixel point in any contour area, when the first target object category to which any pixel point belongs in the second classification result is inconsistent with the object category of the target object to which any pixel point belongs, the first target object category to which any pixel point belongs is corrected to the object category of the target object to which any pixel point belongs; when the number of pixel points of the first target object category consistent with the object category of the target object included in any contour area is less than the number corresponding to a preset proportion of the number of all pixel points in any contour area, the first target object category with the largest number of pixel points belonging to the first target object category among at least one first target object category of multiple pixel points in any contour area is determined as the second target object category of multiple pixel points in any contour area.

[0114] As another correction manner, for any contour area among the multiple contour areas, at least one first target object category of multiple pixel points in any contour area is corrected to the object category of the target object in any contour area.

[0115] In the above implementation, point cloud data and image data are fused, and different classification algorithm schemes are designed in combination with multiple input feature fusions to perform land use classification based on pixel level and using multiple machine learning algorithms, and the influence of single data source classification scheme and multi-source data fusion classification scheme on the classification accuracy of land objects is analyzed, and the influence of single features on classification accuracy and their contribution to classification are compared and analyzed. On the basis of pixel-level classification, image object segmentation technology and multi-classifier integration algorithm are used to realize object-level extraction of land objects.

[0116] Therefore, through the above implementation, based on the feature similarity between the first multi-dimensional feature information of each pixel in the cloud data and the second multi-dimensional feature information of each pixel in the image data, the feature classification of the object category is determined, so that each first classification result is obtained based on the data of multiple data sources of different dimensions, thereby ensuring the accuracy of each first classification result. And each first classification result is evaluated, so that the object category classification result with the best evaluation result is used as the second classification result, so as to reduce the influence of the instability in the application process of each single classification algorithm on the first classification result, thereby ensuring that the second classification result is more accurate. Further, in order to avoid the "pepper and salt" phenomenon in the classification result due to the excessive consideration of the data dimension, that is, the distribution of the two-dimensional and three-dimensional target objects is relatively fragmented, based on the two-dimensional information of the elevation feature information and the spectral feature information, an object-oriented classification method is adopted to further classify the object category of the image, so as to determine the target object to which the multiple pixels included in the contour area belong, and based on the target object classification result, the "pepper and salt" phenomenon problem that may appear in the second classification result is corrected, and finally a more accurate third classification result is obtained.

[0117] The present application also provides the following implementation manner to extract the first multi-dimensional feature information and the second multi-dimensional feature information.

[0118] First, the relative elevation difference between the object and the ground is obtained as elevation feature information based on the normalized digital surface model.

[0119] (Formula 11).

[0120] Among them, nDSM is the relative elevation difference between the object and the ground; DSM is the digital expression of the surface morphology of the objects covered on the surface, such as buildings, vegetation, bridges, etc.; DEM is the ground elevation, which describes the set of spatial positions of surface units and distribution of terrain attributes. It can be defined as a finite vector series in a two-dimensional area, using discretely distributed plane points to simulate continuously distributed terrain.

[0121] Pixels can also be called pixels.

[0122] In some embodiments, the point cloud data is rasterized using the Binning method in the interpolation method. The Binning method not only provides an allocation method for determining each output pixel point using points falling within its range, but also provides a void filling method to determine the pixel value that does not contain any LAS point.

[0123] Firstly, the elevation information of ground point data was extracted, and the pixel value was determined by using the inverse distance weighted (IDW) interpolation algorithm. The natural field method was selected as the void filling method, and the data was resampled into DEM raster data with a resolution of 1 m.

[0124] Then, in the same way, all point cloud data are used as input data to extract their elevation information, and the Binning method is used again to obtain DSM raster data with a preset resolution.

[0125] Finally, the nDSM grid data is obtained using the grid calculator according to the above formula 11, and its coordinate system is defined as UTM 48N / WGS 84.

[0126] Second, the intensity feature information is determined based on the intensity model.

[0127] Given that the laser radar intensity data is related to the energy of the received echo, for discrete echo type laser radar sensors, the laser intensity records the peak amplitude in the backscattered laser beam returned from the object, which is based to a certain extent on the reflectivity of the object scanned by the laser radar pulse. Therefore, each echo of the laser radar scanner that can be used records the intensity information of the preset high resolution (such as 16-bit high resolution), so the intensity can be output as the attribute information of the point cloud itself. In addition, all the denoised point clouds are used as input data to extract their intensity information, which is conducive to the IDW interpolation algorithm to determine the pixel value. The natural field method is selected as the empty filling method, and it is resampled to the intensity model (Intensity Model, IM) of the preset resolution (such as 1m resolution), and its coordinate system is defined as UTM 48N / WGB 84.

[0128] Third, the degree of undulation of the ground object on the surface in the spatial geometric feature information, that is, the roughness, is determined based on the roughness model.

[0129] Although most of the objects can be distinguished based on elevation differences, such as buildings and roads, vegetation and bare soil, there are still some objects that are difficult to distinguish completely based on elevation differences alone, such as buildings and trees, which often have similar elevations and are often distributed together. Therefore, in order to express the differences in surface fluctuations between different objects, a variable is used below to describe and quantify the surface fluctuations of ground objects. Considering that the density of the point cloud is large enough to reflect the surface fluctuation degree of ground objects with the surface roughness coefficient (SRC), SRC can be defined as the ratio of the surface area of ​​the ground object to its projected area in the grid unit, as shown in the following formula.

[0130] (Formula 12).

[0131] in, represents the roughness coefficient of the i-th row and j-th column; and Represents the surface area and projection area of ​​the object in the i-th row and j-th column respectively. The larger the SRC, the greater the surface fluctuation within the unit grid, and the more likely it is that the type of landform has large surface fluctuations such as trees.

[0132] Specifically, first, this application defines the size of the grid as 1m by 1m, and obtains the point cloud data in the cell grid; secondly, polygons are constructed for the points in each grid; next, the surface area of ​​the polygon in each grid is calculated, and the corresponding horizontal projection area is found according to the positions of the four vertices and the projection area is calculated, and the SRC values ​​of all grids are calculated using formula 12; finally, the IDW interpolation algorithm and the natural domain void filling method are used to generate the SRC into a roughness model (Roughness Model, RM) with a resolution of 1m.

[0133] Fourthly, the flatness of the surface of the object in the spatial geometric feature information, namely, the flatness, is determined based on the flatness model.

[0134] Flatness is used to describe the flatness of the surface of an object and is a measure of the surface changes of an object. Generally speaking, buildings, roads, etc. are considered to be composed of planes, while vegetation appears as irregular surfaces. This phenomenon can also be seen from the distribution diagram of each object point cloud. Therefore, the LiDAR points of buildings usually have the characteristic of local flatness, that is, the points are distributed in a planar manner in the local area, while the points on vegetation are mostly non-planar. The following uses the characteristics of covariance analysis to estimate the flatness of the LiDAR point cloud.

[0135] definition represents non-ground points, is the sample point, represent The set of k points closest to the point (the optional k is set to 15 based on experience). of Covariance matrix It can be defined as the following formula.

[0136] (Formula 13).

[0137] in, yes The center point of For collection The number of points.

[0138] Covariance matrix The characteristic value of , , It can be calculated by eigendecomposition, then the point Flatness It can be calculated by the following formula.

[0139] (Formula 14).

[0140] Calculate for all points After that, it is rasterized, and the average interpolation algorithm is used to determine the grid pixel value. The natural field method is used to fill the area without points. Finally, the points containing the flatness attribute value are interpolated and sampled into a flatness model (FTM) with a preset resolution (e.g., 1m resolution).

[0141] Fifth, the degree of discreteness, that is, the degree of dispersion, of the point cloud in the spatial geometric feature information is determined based on the normal difference model.

[0142] Normal variance is a local geometric description based on normal vectors and a measure of the dispersion of normal vectors. Normal lines in vegetation areas are mostly scattered and irregular in direction, while normal lines in areas such as buildings or roads are concentrated in a fixed direction.

[0143] The calculation steps of the normal variance feature mainly include the following steps: (1) Based on the angle between the normal and the vertical direction Constructing a point set histogram (2)Use the square of the coefficient of variation of the histogram As an indicator of the dispersion of normal frequency counts around the average frequency count. Since the normal vectors are clustered in a fixed direction, the point set in the area such as buildings or roads Angle The distribution is very different from the average distribution histogram, while the vegetation is just the opposite. Therefore, in the above steps, All points according to the angle Divided into different direction intervals. of The calculation of is shown in the following formula.

[0144] (Equation 15).

[0145] In the above formula, is the dispersion of the normal frequency counts around the mean frequency count, n is the number of directions, is the number of points in each direction, For collection The number of points, The smaller, the more The closer the distribution histogram is to the average, the greater the probability is that it is in the vegetation area.

[0146] Optionally, for each point That Set to 60, n has little effect on the result, it can be set between 5-10, specifically 6. Calculate all points After that, it is rasterized and the average interpolation algorithm and natural domain void filling method are also used to finally obtain the Variable of Normal Direction Model (VND) with a preset resolution (such as 1m resolution).

[0147] Sixth, extraction of texture feature information.

[0148] Texture feature information is used to describe the local patterns and arrangement rules that recur in an image. The spatial correlation characteristics of grayscale represented by the Gray Level Co-occurrence Matrix (GLCM) are used to describe texture features, so as to reflect the comprehensive information of the image, such as interval, direction, change amplitude, speed, etc., based on the correlation between the grayscales of two points at a certain distance and direction in the image.

[0149] In optical remote sensing images, GLCM is used as input features to extract different types of objects. Specifically, GLCM features are generated using QuickBird images to extract different types of vegetation; GLCM features are generated using SPOT-5 images to extract urban objects. Texture features are extracted based on GLCM using Landsat8 images to map the spatial distribution of artificial deciduous forests. The texture features of LiDAR data are introduced to classify and extract objects. GLCM texture information can be generated based on LiDAR intensity information to facilitate land use classification in an object-oriented manner. GLCM texture features can be calculated based on DSM information extracted from LiDAR and fused with aerial images for target classification. nDSM can be extracted from LiDAR data, GLCM texture information can be calculated based on nDSM, and fused with GLCM generated from high-resolution remote sensing images to achieve research on land use classification. nDSM features can also be generated based on LiDAR, and its GLCM texture information can be further calculated to extract urban building targets.

[0150] Images are also called images.

[0151] Optionally, eight features including mean, variance, homogeneity, contrast, dissimilarity, information entropy, second-order moment and correlation are used as texture feature information. It can be understood that the GLCM texture features are calculated based on the nDSM elevation information and the second-order probability statistics method. The second-order probability statistics uses a grayscale spatial correlation matrix to calculate the texture value, which is a relative frequency matrix, that is, the frequency of pixel values ​​appearing in two adjacent processing windows separated by a specific distance and direction. The matrix shows the number of occurrences of the relationship between a pixel and its specific neighborhood as texture feature information. Among them, the texture feature information corresponding to the mean feature is the grayscale average of the pixels in the moving window; the texture feature information corresponding to the variance feature is the variance of the pixels in the window. The larger the value, the larger the grayscale change in the current range; the texture feature information corresponding to the homogeneity feature is a description of the similarity of image texture features. The larger the value, the smaller the texture change in the current window range, and the smaller the grayscale difference; the texture feature information corresponding to the contrast feature is a description of the change in grayscale in the field. The larger the value, the greater the image change in the current area; the texture feature information corresponding to the dissimilarity feature is a description of the contrast of grayscale in the field. The larger the value, the greater the contrast; the texture feature information corresponding to the information entropy is a measure of the amount of information in the image. The larger the value, the more obvious the heterogeneity of the current region and the more complex the image; the texture feature information corresponding to the second-order moment is a measure of the uniformity of image grayscale distribution and texture coarseness. The larger the value, the more delicate the image texture and the more uniform the grayscale distribution; the texture feature information corresponding to the correlation is a description of the linear correlation of image grayscale. The larger the value, the higher the texture homogeneity.

[0152] In the process of calculating GLCM texture features, the selection of moving window size is an important parameter, which not only affects the resolution of texture features but also is very important for the characterization of image edges. You can choose a preset number of windows of different sizes for calculation, compare the 8 texture feature information obtained under different windows, sum the 8 texture feature information, and take the moving window of the 8 texture feature information with the largest sum as the best moving window.

[0153] Exemplarily, five windows of different sizes, 3*3, 5*5, 7*7, 9*9, and 11*11, are selected, nDSM is used as the input object, and GLCM texture is used for calculation, and a 3*3 (i.e., 3 times 3) moving window size is selected, and the transformation value is set to 1 for X and 1 for Y, and 8 texture features are extracted, which is the best effect.

[0154] Seventh, extraction of spectral feature information.

[0155] The visible light vegetation index is extracted using the three bands R, G, and B of airborne aerial images. First, the three original spectral bands R, G, and B are processed and their pixel values ​​are normalized to the range of 0-1. The calculation formula is as follows.

[0156] (Equation 16).

[0157] In the above formula, R, G, and B are the three original red, green, and blue spectral bands of the aerial remote sensing image, respectively, and r, g, and b are the normalized results of the red, green, and blue spectral bands, respectively.

[0158] The spectral characteristic information includes the visible light vegetation index.

[0159] There are eight commonly used visible vegetation indices, including Normalized Green-Red difference index (NGRDI), Excess Green Index (EXG), Color Index of Vegetation (CIVE), Vegetation Index (VEG), Excess Green Minus Excess Red Index (EXGR), Woebbecke Index (WI), Combination Index 1 (COM1) and Combination Index 2 (COM2).

[0160] Eighth, extraction of morphological building index.

[0161] The Morphological Building Index (MBI) is to establish the correspondence between the intrinsic characteristics of buildings, such as brightness, size and contrast, and morphological operators such as top hat reconstruction, granularity analysis and directionality. The basis is that the spectral characteristics inside the building vary little, while the spectral characteristics at its edge vary greatly. The MBI calculation method can be used to establish the MBI based on the intrinsic characteristics of the building, such as brightness, contrast, size, shape and direction, to improve the distinction between buildings and roads, bare soil, etc.

[0162] The first is brightness. Since the visible light band has a greater impact on the spectral information of the building and has a higher reflectivity on the roof of the building, the maximum value of the visible light band of each pixel is used as the brightness value of the corresponding pixel. Morphological white hat reconstruction is used to display bright structures with brightness reaching a predefined value.

[0163] Secondly, local contrast. The roof of a building has a relatively high reflectivity, and the shadow area adjacent to it causes a high local contrast. The differential morphological profile is used to characterize the local contrast of the bright structure. In addition, size and directionality. Buildings and roads have similar spectral information due to material and other reasons, but buildings have more directions, while roads have fewer directions, generally extending in one or two directions. Therefore, buildings and roads can be better distinguished based on this characteristic, and MBI can be selected on a series of linear structural elements that can measure the size and directionality of the structure.

[0164] Finally, shape, the shape features of buildings such as rectangle, aspect ratio, etc. can also be used as constraints to remove narrow and elongated structures.

[0165] The specific implementation process of MBI extraction is as follows.

[0166] Step 1: Calculate the brightness value.

[0167] (Equation 17).

[0168] In the formula, Pixel The brightness value, Pixel exist The spectral value of the band, K is the total number of spectral bands, and K can be 3 in the embodiment of the present application.

[0169] Step 2: White top-hat (W-TH) reconstruction.

[0170] (Equation 18).

[0171] In the formula, Indicates the morphological reconstruction of the brightness value of the image, b is the brightness value of the pixel, Respectively represent the direction and size of the linear structure element.

[0172] Step 3: Calculate morphological profiles (MP).

[0173] (Equation 19).

[0174] Step 4: Calculate differential morphological profiles (DMP).

[0175] (Equation 20).

[0176] In the formula, is the interval of the profile and .

[0177] Step 5: Calculate MBI.

[0178] (Formula 21).

[0179] In the formula, D and S represent the direction and size of the section respectively. The direction can be set to 4 ; Linear structural element , and The size is determined by the spatial characteristics of the building and the spatial resolution of the image, and can be set to , and ;also, .

[0180] The above MBI is established based on the fact that building structures have larger values ​​in most directions of the white hat DMP histogram, thus forming a higher local contrast. The larger the MBI value, the greater the possibility that the area is a building.

[0181] Eighth, extraction of morphological shadow index.

[0182] The Morphological Shadow Index (MSI) is based on the spectral characteristics of shadows, and fully considers structural features such as brightness, shape, size, contrast and direction, and establishes a corresponding relationship between it and morphological operators such as top hat reconstruction, granularity analysis and directionality. MSI can be regarded as a twin index of MBI, because shadows show characteristics that are spectrally opposite to buildings but spatially similar. First, brightness. In the visible light band, shadows show low reflectivity, so a threshold can be predefined to extract the brightness of the shadow structure; second, local contrast. Shadows also show high local contrast, but unlike buildings, shadow structures are obviously darker than adjacent structures, so morphological black hat transformation and differential morphological profiles can be used to represent the local contrast of shadows; in addition, size and direction. The direction of the shadow is related to the position of the sun, and the size of the shadow is affected by factors such as the height and size of the building, the height of the sun, and the distance between buildings. Based on the above analysis, the calculation of MSI can be expanded according to MBI, that is, the white hat transformation in MBI is replaced by the morphological black hat transformation. The specific implementation steps are as follows.

[0183] Step 1: Calculate the brightness value using the same method as formula 17.

[0184] Step 2: Black top-hat (B-TH) reconstruction.

[0185] (Formula 22).

[0186] In the formula, It represents the morphological closing operation on the brightness value of the image. d and s represent the direction and size of the linear structure element respectively.

[0187] Step 3: Calculation of Morphological Profiles (MP)

[0188] (Formula 23).

[0189] Step 4: Calculate differential morphological profiles (DMP)

[0190] (Equation 24).

[0191] In the formula, is the interval of the section, .

[0192] Step 5: Calculate MSI.

[0193] (Equation 25).

[0194] The shadow area has a larger MSI value, which appears as a larger brightness value in the MSI feature image. Through MSI calculation, some bare soil, roads and other objects with larger brightness values ​​can be distinguished from buildings with similar spectral characteristics based on MBI. On the other hand, for some building roofs with large heterogeneity or darker colors, MSI can be used as a spatial constraint to further reduce the misclassification phenomenon.

[0195] Ninth, the extraction of urban complexity.

[0196] The growth of urbanization affects various complex natural and socio-economic phenomena. Quantitative estimation of the standardized level of urbanization is crucial for analyzing the relationship between urbanization and other factors. Urban complexity is a three-dimensional discrete wavelet transform (3D-DWT) constructed based on the spatial characteristics, spectral characteristics and energy functions of images. The basic idea is that in spatial images, remote sensing images covering cities and remote sensing images covering suburbs show different spatial and spectral characteristics. For example, the spectral characteristics of natural objects such as water, trees, grass and bare soil vary greatly, while the spatial information shows slight changes. However, the spatial characteristics of urban areas composed of buildings, roads and transportation facilities vary more than the spectral information. 3D-DWT extends the one-dimensional wavelet transform to multi-channel data or volume data. It can distinguish complex urban areas from other areas, and combine these spectral and spatial attributes to express digital complexity standards, which can effectively analyze spatial and spectral information.

[0197] Wavelet transform uses a set of functions with local properties in both time domain and frequency domain to decompose the signal, and uses wavelet transform as basis function to represent and analyze the given target function. Wavelet transform can be composed of a series of scaling functions. and wavelet function composition.

[0198] (Equation 26).

[0199] Where j and k represent the scaling parameter and translation parameter respectively.

[0200] The above scaling function can be a low-pass filter, and the wavelet function Could be a high pass filter.

[0201] 3D-DWT is an extension of wavelet theory from one-dimensional signals to three-dimensional signals. For remote sensing images, 3D-DWT not only decomposes the data in the spatial row and column directions, but also decomposes the spectral direction as the depth direction. In terms of wavelet space decomposition, similar to the two-dimensional discrete wavelet transform, 3D-DWT can be constructed using tensor product through the following formula.

[0202] (Equation 27).

[0203] In the formula, and are spatial addition and multiplication respectively; L and H are low-pass filter and high-pass filter along the x, y and z axes respectively, the x and y directions are the spatial coordinates of the image, and the z direction is the spectral axis.

[0204] Based on the calculation of Formula 27, the first-level 3D-DWT decomposition of the image cube produces a total of 8 sub-bands: LLL, LLH, LHL, LHH, HLL, HLH, HHL, HHH. For example, LLH represents a sub-band of low-frequency information in the spatial row and column directions, and a sub-band of high-frequency information in the spectral direction. In the experiment, the 8 sub-bands include the following 8 wavelets, which are obtained by a one-dimensional scaling function and the one-dimensional wavelet function It is constructed by the tensor product of .

[0205] (Formula 28).

[0206] These eight sub-bands can be divided into three categories: the first is the approximate component (LLL) and corner point (HHH) of the model; the second represents the change of spectrum (**H), namely LLH, LHH, HLH; the third represents the change of space (**L), namely LHL, HHL, HLL. **H represents the change of spectrum caused by the high-pass filter in the spectral direction; **L represents the change of space, which represents the horizontal and vertical information in the spatial domain and the low-frequency information in the spectral domain.

[0207] Therefore, UCI is defined as the sum of all H components (**H) divided by the sum of all L components.

[0208] (Formula 29).

[0209] Here, the function E(*) represents the energy function of the sub-band, which is calculated as follows.

[0210] (Formula 30).

[0211] Where i, j and m represent the coordinates of the x, y and z directions in the sub-band f, respectively.

[0212] The calculation of the above UCI is based on a moving window centered on each pixel in the image cube. Therefore, the selection of window size becomes a key issue when applying UCI to feature extraction and classification of remote sensing images. According to the characteristics of high-resolution aerial images, a 4x4 window can be selected for UCI feature extraction. In the UCI feature image, buildings, roads, etc. have larger brightness values ​​than natural landscapes such as trees, grass, water, and soil.

[0213] In order to achieve the above functions, the land use category classification device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0214] The present application also provides a method Figure 3 The land use category classification device shown includes: an acquisition unit 301, a first classification unit 302, an evaluation unit 303, a second classification unit 304 and a third classification unit 305.

[0215] The acquisition unit 301 is used to acquire point cloud data from a laser radar scanner of an airborne LiDAR measurement system and image data from an airborne digital camera; the resolution of the airborne digital camera is higher than a resolution threshold.

[0216] The first classification unit 302 is used to adopt multiple different land use classification models, and classify the land object category to which each pixel point belongs according to the feature similarity between the first multi-dimensional feature information of each pixel point in the point cloud data and the second multi-dimensional feature information of each pixel point in the image data being higher than the corresponding similarity threshold, so as to obtain multiple first classification results.

[0217] The evaluation unit 303 is used to evaluate and analyze the ground object category to which each pixel point in the multiple first classification results belongs according to the frequency of the ground object category to which each pixel point in the multiple first classification results belongs, and determine the ground object category to which each pixel point with the highest evaluation result belongs as the first target ground object category of each pixel point, and obtain the second classification result including the first target ground object category to which each pixel point belongs.

[0218] The second classification unit 304 is used to segment the image including each pixel point according to a preset segmentation scale, and use the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information as a basis for determining the contour features of the object, and based on a preset merging condition, merge the segmented areas in the image to obtain an image including multiple contour areas corresponding to multiple target objects; multiple pixel points included in a contour area belong to one target object.

[0219] The third classification unit 305 is used to correct the first target object category to which the pixel points identical to the pixel points in the contour area in the second classification result belong based on the target object to which the pixel points in the contour area belong, and obtain a third classification result of the second target object category including each pixel point.

[0220] In one embodiment, before adopting multiple different land use classification models, the feature similarity between the first multi-dimensional feature information of each pixel in the point cloud data and the second multi-dimensional feature information of each pixel in the image data is higher than the corresponding similarity threshold, and the land object category to which each pixel belongs is classified. Before obtaining multiple first classification results, the acquisition unit 301 is also used to: extract indirect feature information and direct feature information of each pixel from the point cloud data, and extract spectral feature information and image space feature information of each pixel from the image data; the direct feature information includes elevation feature information and intensity feature information; the indirect feature information includes spatial geometric feature information of the point cloud and texture feature information used to describe the correlation between the grayscale of two points within a preset distance and in a preset direction; the spatial geometric feature information includes the undulation of the land object on the surface, the flatness of the land object surface and the discreteness of the point cloud; the image space feature information includes: morphological building index, morphological shadow index and urban complexity.

[0221] The spectral characteristic information includes the visible light vegetation index.

[0222] In another embodiment, the first classification unit 302 is specifically used to: adopt multiple different land use classification models, classify the land object category to which each pixel point belongs according to the feature similarity between the indirect feature information, direct feature information, spectral feature information and image space feature information of each pixel point being higher than the corresponding similarity threshold, and obtain multiple first classification results.

[0223] In another implementation, the multiple different land use classification models are different machine learning models; the first classification unit 302 is also used to: construct multiple different initial models that characterize the association between each feature information of the point cloud data and the image data and the land feature category to which each pixel point belongs; the multiple initial models include a KNN classifier algorithm model, an RF classifier algorithm model, and an XGBoost classifier algorithm model; extract historical indirect feature information and historical direct feature information of each historical pixel point from multiple groups of historical point cloud data, and extract historical spectral feature information of each historical pixel point from multiple groups of historical image data corresponding to the multiple groups of historical point cloud data; The spatial feature information of historical images is obtained, and multiple groups of historical images corresponding to multiple groups of historical point cloud data are obtained, and the land object category to which each historical pixel point in the historical image belongs has been marked; according to the preset proportion rule, the multiple groups of historical point cloud data, the multiple groups of historical image data and the multiple groups of historical images are selected into training sample data sets and verification sample data sets; the preset proportion rule refers to selecting the number of samples of each land object category according to the proportion of each land object category in the historical image area; multiple initial models are trained separately based on the training sample data set, and the verification sample data set is used to verify each trained initial model at the same time, so that each initial model with the best verification result is used as multiple different land use classification models.

[0224] The historical spectral characteristic information includes the historical visible light vegetation index.

[0225] In another embodiment, the evaluation unit 303 is also used for: for any pixel point among each pixel point, if among the multiple feature categories to which any pixel point in multiple first classification results belongs, the same feature category with the largest number of belongings exists and the same feature category is unique, then the same feature category with the largest number of belongings is determined as the feature category to which any pixel point with the highest evaluation result belongs; if among the multiple feature categories to which any pixel point in multiple first classification results belongs, the same feature category with the largest number of belongings exists and the same feature category is greater than or equal to 2, then the same feature category with the highest priority in the first classification results to which the same feature category with the largest number of belongings exists is determined as the feature category to which any pixel point with the highest evaluation result belongs; wherein the priorities of the multiple first classification results are positively correlated with the priorities of the corresponding multiple different land use classification models; the priorities of the multiple different land use classification models are determined based on the adaptability of the first multi-dimensional feature information and the second multi-dimensional feature information to the land use classification model or the influencing factor on the land use classification model.

[0226] In another implementation, the third evaluation unit 305 is specifically used to: when the number of pixel points of the first target object category consistent with the object category of the target object included in any contour area is greater than or equal to the number corresponding to the preset proportion of the number of all pixel points in any contour area, for any pixel point in each pixel point in any contour area, when the first target object category to which any pixel point belongs in the second classification result is inconsistent with the object category of the target object to which any pixel point belongs, correct the first target object category to which any pixel point belongs to be the object category of the target object to which any pixel point belongs; when the number of pixel points of the first target object category consistent with the object category of the target object included in any contour area is less than the number corresponding to the preset proportion of the number of all pixel points in any contour area, determine the first target object category with the largest number of pixel points belonging to the first target object category among at least one first target object category of multiple pixel points in any contour area as the second target object category of multiple pixel points in any contour area; and / or, for any contour area among multiple contour areas, correct at least one first target object category of multiple pixel points in any contour area to the object category of the target object in any contour area.

[0227] In another embodiment, the preset merging conditions include one or more of the following: minimizing object heterogeneity or minimizing image smoothness heterogeneity, minimizing image compactness heterogeneity, minimizing image shape heterogeneity, and minimizing the average heterogeneity of all objects; the preset segmentation scale is determined according to the object type of the segmented object.

[0228] In another embodiment, the point cloud data and the image data are respectively corrected point cloud data and image data; the acquisition unit 301 is also used to: acquire the attitude information and position information of the aircraft from the positioning device of the airborne LiDAR measurement system; the positioning device includes an IMU (Inertial Measurement Unit) inertial measurement unit and a differential GPS (Differential Global Positioning System) differential global positioning system; based on the attitude information and position information of the aircraft, the point cloud data and the image data are respectively corrected to obtain the corrected point cloud data and image data; the elevation of the corrected point cloud data is within a preset elevation range, and the average point cloud density of the point cloud data is within a preset density range; the horizontal resolution of the point cloud data is within a first preset resolution; the vertical resolution of the point cloud data is within a second preset resolution; the horizontal resolution and vertical resolution of the image data are the same and are both within a third preset resolution; the width of the image data is within a preset width.

[0229] Regarding the device in the above embodiment, the specific manner in which each unit module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0230] Figure 4 Schematic diagram of a computer device provided by this application. Figure 4 The computer device 60 may include at least one processor 601 and a memory 603 for storing processor executable instructions. The processor 601 is configured to execute instructions in the memory 603 to implement the land use category classification method in the following embodiment.

[0231] In addition, the computer device 60 may also include a communication bus 602 , at least one communication interface 604 , an input device 606 , and an output device 605 .

[0232] The processor 601 may be a central processing unit (CPU), a microprocessing unit, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application.

[0233] Communication bus 602 may include a pathway for transmitting information between the above-mentioned components.

[0234] The communication interface 604 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0235] The input device 606 is used to receive input signals and the output device 605 is used to output signals.

[0236] The memory 603 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processing unit through a bus. The memory may also be integrated with the processing unit.

[0237] The memory 603 is used to store instructions for executing the solution of the present application, and the execution is controlled by the processor 601. The processor 601 is used to execute the instructions stored in the memory 603, so as to realize the functions in the method of the present application.

[0238] In a specific implementation, as an embodiment, the processor 601 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 in.

[0239] In a specific implementation, as an embodiment, the computer device 60 may include multiple processors, such as Figure 4 601 and processor 607 in the embodiment of the present invention. Each of these processors may be a single-CPU processor or a multi-CPU processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0240] The computer equipment Figure 4 The system includes: a processor 601 and a memory 603 for storing executable instructions of the processor 601; wherein the processor 601 is configured to execute the executable instructions to implement a land use classification method as described in any possible implementation method described above. The same technical effect can be achieved, and to avoid repetition, it will not be described here.

[0241] The embodiment of the present application also provides a computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by the processor of the control device or control equipment, the control device or control equipment can execute the land use category classification method as any possible implementation method described above. And can achieve the same technical effect, to avoid repetition, it will not be repeated here.

[0242] The present application also provides a computer program product, including a computer program or an instruction, which is executed by a processor as a land use classification method in any of the above possible implementations. The same technical effect can be achieved, and it will not be repeated here to avoid repetition.

[0243] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0244] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A land use classification method, characterized in that: The method comprises: Acquire point cloud data from a laser radar scanner of an airborne LiDAR measurement system, and image data from an airborne digital camera; the resolution of the airborne digital camera is higher than a resolution threshold; and extract indirect feature information and direct feature information of each pixel from the point cloud data, and extract spectral feature information and image space feature information of each pixel from the image data; the direct feature information includes elevation feature information and intensity feature information; the indirect feature information includes spatial geometric feature information of the point cloud and texture feature information for describing the correlation between the grayscale of two points within a preset distance and in a preset direction; the spatial geometric feature information includes the undulation of the object on the surface, the flatness of the surface of the object, and the discreteness of the point cloud; the image space feature information includes: morphological building index, morphological shadow index and urban complexity; Using a plurality of different land use classification models, according to the feature similarity between the first multi-dimensional feature information of each pixel point in the point cloud data and the second multi-dimensional feature information of each pixel point in the image data being higher than the corresponding similarity threshold, classifying the land object category to which each pixel point belongs, and obtaining a plurality of first classification results; the land use classification model represents the association relationship between each feature information of the point cloud data and the image data and the land object category to which each pixel point belongs; According to the frequencies of the ground object categories to which the respective pixel points belong in the plurality of first classification results, the ground object categories to which the respective pixel points belong in the plurality of first classification results are evaluated and analyzed, and the ground object category to which the respective pixel points with the highest evaluation result belong is determined as the first target ground object category of the respective pixel points, to obtain a second classification result including the respective first target ground object categories to which the respective pixel points belong; According to a preset segmentation scale, the image including the pixels is segmented, and the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information are used as the basis for determining the contour features of the object, and based on a preset merging condition, the segmented regions in the image are merged to obtain the image including a plurality of contour regions corresponding to a plurality of target objects; the plurality of pixels included in one contour region belong to one target object; Based on the target object to which the pixel points in the contour area belong, correcting the first target object category to which the pixel points identical to the pixel points in the contour area in the second classification result belong, to obtain a third classification result of the second target object category including each pixel point; The method adopts multiple different land use classification models, and classifies the land object category to which each pixel point belongs according to the feature similarity between the first multi-dimensional feature information of each pixel point in the point cloud data and the second multi-dimensional feature information of each pixel point in the image data being higher than the corresponding similarity threshold, to obtain multiple first classification results, including: adopting the multiple different land use classification models, and classifying the land object category to which each pixel point belongs according to the feature similarity between the indirect feature information, direct feature information, spectral feature information, visible light vegetation index and image space feature information of each pixel point being higher than the corresponding similarity threshold, to obtain the multiple first classification results.

2. The method according to claim 1, characterized in that The multiple different land use classification models are different machine learning models; the method further comprises: Constructing a plurality of different initial models representing the association between each feature information of the point cloud data and the image data and the ground object category to which each pixel point belongs; the plurality of initial models include a K nearest neighbor classifier algorithm model, a random forest classifier algorithm model and an extreme gradient boosting classifier algorithm model; Extracting historical indirect feature information and historical direct feature information of each historical pixel point from multiple sets of historical point cloud data, and extracting historical spectral feature information and historical image spatial feature information of each historical pixel point from multiple sets of historical image data corresponding to the multiple sets of historical point cloud data, and obtaining multiple sets of historical images corresponding to the multiple sets of historical point cloud data, wherein the ground object category to which each historical pixel point belongs in the historical image has been marked; According to a preset ratio rule, the plurality of groups of historical point cloud data, the plurality of groups of historical image data and the plurality of groups of historical images are selected into training sample data sets and verification sample data sets; the preset ratio rule refers to selecting the number of samples of each ground object category according to the ratio of each ground object category in the historical image area; The multiple initial models are trained separately based on the training sample data set, and the validation sample data set is used to validate each trained initial model, so that each initial model with the best validation result is used as the multiple different land use classification models.

3. The method according to claim 1 or 2, characterized in that: The evaluating and analyzing the ground object category to which each pixel point in the plurality of first classification results belongs according to the frequency of the ground object category to which each pixel point in the plurality of first classification results belongs includes: For any pixel point among the pixels, if among the multiple ground object categories to which the pixel point belongs in the multiple first classification results, there is a ground object category to which the number of belongings is the largest and the same ground object category is unique, then the same ground object category to which the number of belongings is the largest is determined as the ground object category to which the pixel point with the highest evaluation result belongs; If the number of the same land feature category to which any pixel point belongs in the multiple land feature categories among the multiple first classification results is the largest and the number of the same land feature categories is greater than or equal to 2, then the same land feature category with the highest priority among the first classification results of the multiple same land feature categories with the largest number of belonging is determined as the land feature category to which any pixel point with the highest evaluation result belongs; wherein the priorities of the multiple first classification results are positively correlated with the priorities of the corresponding multiple different land use classification models; and the priorities of the multiple different land use classification models are determined based on the adaptability of the first multi-dimensional feature information and the second multi-dimensional feature information to the land use classification model or the influencing factors of the land use classification model.

4. The method according to claim 1 or 2, characterized in that: The method of correcting the first target object category to which the pixel points in the second classification result, which are the same as the pixel points in the contour area, belong based on the target object to which the pixel points in the contour area belong, to obtain a third classification result of the second target object category including each pixel point, comprises: When the number of pixel points of the first target object category consistent with the object category of the target object included in any of the contour areas is greater than or equal to the number corresponding to the preset proportion of the number of all pixel points in any of the contour areas, for any pixel point among the individual pixel points in any of the contour areas, when the first target object category to which the any pixel point belongs in the second classification result is inconsistent with the object category of the target object to which the any pixel point belongs, the first target object category to which the any pixel point belongs is corrected to the object category of the target object to which the any pixel point belongs; When the number of pixel points of the first target object category consistent with the object category of the target object included in any of the contour areas is less than the number corresponding to the preset proportion of the number of all pixel points in any of the contour areas, the first target object category with the largest number of pixel points belonging to the first target object category among at least one first target object category of multiple pixel points in any of the contour areas is determined as the second target object category of multiple pixel points in any of the contour areas; and / or For any contour area among the multiple contour areas, at least one first target object category of multiple pixel points in the any contour area is corrected to the object category of the target object in the any contour area.

5. The method according to claim 1 or 2, characterized in that: The preset merging conditions include one or more of the following: minimizing object heterogeneity or minimizing image smoothness heterogeneity, minimizing image compactness heterogeneity, minimizing image shape heterogeneity, and minimizing the average heterogeneity of all objects; the preset segmentation scale is determined according to the object type of the segmented object.

6. The method according to claim 1 or 2, characterized in that: The point cloud data and the image data are respectively the corrected point cloud data and the image data; the method further comprises: Acquire attitude information and position information of the aircraft from a positioning device of the airborne LiDAR measurement system; the positioning device includes an IMU inertial measurement unit and a differential global positioning system; The point cloud data and the image data are corrected respectively based on the attitude information and the position information of the aircraft to obtain the corrected point cloud data and the image data; the elevation of the corrected point cloud data is within a preset elevation range, and the average point cloud density of the point cloud data is within a preset density range; the horizontal resolution of the point cloud data is within a first preset resolution; the vertical resolution of the point cloud data is within a second preset resolution; the horizontal resolution and the vertical resolution of the image data are the same and are both within a third preset resolution; the width of the image data is within a preset width.

7. A land use classification device, characterized in that: The device comprises: An acquisition unit is used to acquire point cloud data from a laser radar scanner of an airborne LiDAR measurement system, and image data from an airborne digital camera; the resolution of the airborne digital camera is higher than a resolution threshold; and, extract indirect feature information and direct feature information of each pixel point from the point cloud data, and, extract spectral feature information and image space feature information of each pixel point from the image data; the direct feature information includes elevation feature information and intensity feature information; the indirect feature information includes spatial geometric feature information of the point cloud and texture feature information for describing the correlation between the grayscale of two points within a preset distance and in a preset direction; the spatial geometric feature information includes the undulation degree of the object on the surface, the flatness of the surface of the object and the discreteness of the point cloud; the image space feature information includes: morphological building index, morphological shadow index and urban complexity; A first classification unit is used to adopt a plurality of different land use classification models, and classify the land object category to which each pixel point belongs according to the feature similarity between the first multi-dimensional feature information of each pixel point in the point cloud data and the second multi-dimensional feature information of each pixel point in the image data being higher than the corresponding similarity threshold, to obtain a plurality of first classification results; the land use classification model represents the association relationship between each feature information of the point cloud data and the image data and the land object category to which each pixel point belongs; An evaluation unit, configured to evaluate and analyze the ground object category to which each pixel point in the plurality of first classification results belongs according to the frequency of the ground object category to which each pixel point in the plurality of first classification results belongs, and determine the ground object category to which each pixel point with the highest evaluation result belongs as the first target ground object category of each pixel point, and obtain a second classification result including each first target ground object category to which each pixel point belongs; A second classification unit is used to segment the image including the pixels according to a preset segmentation scale, and to use the elevation feature information in the first multi-dimensional feature information and the spectral feature information in the second multi-dimensional feature information as a basis for determining the contour features of the object, and to merge the segmented regions in the image based on a preset merging condition to obtain the image including a plurality of contour regions corresponding to a plurality of target objects; the plurality of pixels included in one contour region belong to one target object; The third classification unit is used to correct the first target object category to which the pixel points in the second classification result that are the same as the pixel points in the contour area belong based on the target object to which the pixel points in the contour area belong, so as to obtain a third classification result of the second target object category including each pixel point The first classification unit is specifically used to: adopt the multiple different land use classification models, and classify the land object category to which each pixel point belongs according to the feature similarity between the indirect feature information, direct feature information, spectral feature information, visible light vegetation index and image space feature information of each pixel point being higher than the corresponding similarity threshold, to obtain the multiple first classification results.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor, the land use category classification method according to any one of claims 1 to 6 can be performed.

Images