Intelligent classification method for artificial building scenes in high-resolution remote sensing images

Abstract

The intelligent classification method for artificial building scenes in high-resolution remote sensing images of this application establishes a vision-oriented hidden layer probabilistic semantic analysis scene classification method, adds spatial information constraints to the features of scene expression, carries more semantic information with lower feature dimensions, and improves the expressive power of visual vocabulary. First, the image performance characteristics of typical artificial scenes are analyzed in combination with the data features of remote sensing images, and a lower feature dimension is used to describe the target visual information; second, the hidden layer probabilistic semantic analysis model is improved in combination with the features of remote sensing images, and the strategy of segmenting the most characteristic vocabulary of vision is adopted to describe the visual features of the over-segmented image, and the rich image spatial information and geometric structure information brought by high spatial resolution are used to make up for the limited description power of local feature points; by combining the vision-oriented classification method with the focus model, a better classification effect is achieved with low-dimensional feature description.

Description

Technical Field

[0001] The present application relates to a forestry remote sensing image scene classification method, and in particular to an intelligent classification method for artificial building scenes in high-resolution remote sensing images, and belongs to the technical field of remote sensing image classification. Background Art

[0002] High spatial resolution remote sensing images contain a large amount of intuitive spatial information, with clear and obvious structures and rich texture information, which provide sufficient data support for accurate target recognition and scene interpretation. Scene recognition technology is one of the important tasks of remote sensing image understanding. Under the current situation that manual interpretation methods cannot meet the efficiency and real-time requirements, the automatic discovery and understanding of artificially constructed targets and scene contents have gradually received attention and research in recent years, and have broad application prospects in forestry resource surveys, urban planning, municipal transportation and other fields. Scene classification is a high-level image processing method. Since the semantic content of the image cannot be accurately described by the underlying features, the conversion from data to knowledge is fragmented, which is particularly evident in remote sensing images.

[0003] Scene classification is the process of abstracting high-level information in an image into semantic labels. With the innovation of machine vision technology, scene classification technology has been widely used in computer vision, artificial intelligence, military, forestry, etc. In the field of remote sensing, with the increase in the number of images and the need for interpretation, the classification method that relies on manual vision is challenged. Existing forestry surveys and other technologies cannot automatically and accurately complete scene classification.

[0004] The commercial development of high-resolution remote sensing has made the application of high-resolution remote sensing images no longer limited to forestry surveys, environmental monitoring, land use, and surveying and mapping. The development of the Internet has also prompted high-resolution remote sensing technology to open up many new application areas such as electronic maps, tourism, and intelligent transportation. Recognizing remote sensing image scenes can better provide convenient and effective information for people's travel and help other industries obtain information. Applications include: resource surveys, specific target identification, response to earthquakes and landslides, image intelligent retrieval and query technology, etc.

[0005] The problems that need to be solved in the existing high-resolution remote sensing image scene classification and the key technical difficulties of this application include:

[0006] (1) Scene annotation technology for remote sensing images requires advanced information extraction and analysis, focusing more on the overall content of the image and relying on human experience, inference and judgment. There are many problems with the current scene automatic classification method. At present, there is a large demand for remote sensing information, and the data update cycle is getting shorter and more frequent. Manual interpretation and the original processing methods are obviously unable to meet the current application needs. Faced with massive and large-scale image data, information extraction consumes a lot of energy and is difficult to meet the real-time accuracy requirements. How to use computers to efficiently, accurately and intelligently process these remote sensing images and realize the transformation of data into value is an application problem that still needs to be solved at this stage.

[0007] (2) Existing technologies focus more on the detection and recognition of specific single targets in remote sensing images. The expression of remote sensing scenes is quite difficult. First, due to the complex relationship between scenes and targets, the difficulty of dividing scene content is much greater than the recognition of single targets. Secondly, there is no unified standard for the division and definition of scene categories, and there is no universal standard for how to accurately define the boundaries of specific scene targets in large-scale remote sensing images. Rapid retrieval and annotation of a large number of remote sensing images by computers, as well as high-level automatic understanding of scene semantics, are of great application value.

[0008] (3) The demand for automated scene understanding of remote sensing images is becoming increasingly apparent. The existing manual management and labeling methods can no longer meet the requirements of real-time and accuracy. On the one hand, the rapid growth of remote sensing data has caused manual labeling to consume a lot of manpower; on the other hand, manual labeling is subjective and has the possibility of randomness. It is urgent to use computer technology to learn human cognitive methods to automatically classify the semantic categories of remote sensing images. The observation vision of remote sensing images is the surface of the earth. Due to the uncertainty of sensors and imaging conditions, they are inherently complex and contain a large amount of information. Due to the complex changes in surface coverage, the corresponding scenes also have the characteristics of complex targets and complex relationships. Different from the classification tasks of general scenes, the scene recognition and classification of remote sensing images are more difficult. Remote sensing images use computer vision technology for conventional natural image classification. If they are directly used for remote sensing image scene classification, the performance is poor, and the performance is even more general in complex scenes containing multiple categories of objects.

[0009] (4) The existing technology constructs semantic models to perform semantic analysis on images, with the bag-of-words model and probabilistic model being the most widely used. However, this method has certain limitations. First, although the SIFT feature has good invariance to changes in illumination, scale, and angle, the SIFT feature itself is a local feature. As a visual word, it has no actual physical meaning. When dealing with complex remote sensing images containing a variety of landforms, the classification accuracy decreases. Second, this method only infers the scene semantics based on the frequency of word occurrence, ignoring the relationship between words. In addition, the large vocabulary and large dimensionality of high-resolution image features and the time-consuming clustering also limit the efficiency of the algorithm.

[0010] (5) The features and models of existing technologies have their limitations. They can only achieve good recognition and detection effects for a certain type of land object, or can extract special features with good recognition for specific scenes such as airports. It is still a difficult problem to design robust algorithms with high classification accuracy for data sources of different quality and scale. At present, scene classification research has gradually shifted from simple bottom-level feature extraction to mid- and high-level semantic information mining. The mapping of bottom-level local features to mid- and high-level semantics is poor in robustness, and the semantic gap problem is still difficult to cross. The semantic information of local features is insufficient, and it is difficult to grasp the key information of semantic annotation. The current problems are in the following two aspects. First, how to quickly obtain high-level semantic information of image pairs for effective classification and improve the classification accuracy of scenes. Secondly, the processing methods of high-resolution remote sensing data and the improvement of algorithm efficiency cannot be ignored. Summary of the invention

[0011] This application establishes a scene semantic classification method based on segmented vision, improves the hidden layer probability semantic parsing model combined with the local feature descriptor method, provides effective spatial constraint information for remote sensing image scene classification, and improves the scene expression effect and classification accuracy. The first is a mapping method from low-level visual features to high-level semantic features: the high-level semantic expression of an image is inseparable from the underlying features. In view of the semantic gap problem and the spectral variability of the ground object category, a feature description method for vision is established. Different features represent different features of the image, and their extraction methods and efficiencies vary. By selecting a combination of features with strong expression power and strong complementarity, the description accuracy of the segmented target is improved. The second is a high-resolution remote sensing image scene classification method based on hidden layer probability semantic parsing: the probability model and the bag-of-words model are inspired by text processing. The vocabulary expression method using local features does not utilize spatial information and ignores the spatial relationship between visual words. Spatial information is particularly important in remote sensing images. This application uses the segmented image vision as a vocabulary unit, incorporates spatial information into the model, quantifies spatial information, and plays a role in scene recognition, thereby improving recognition accuracy and efficiency.

[0012] In order to achieve the above technical effects, the technical solutions adopted in this application are as follows:

[0013] An intelligent classification method for artificially constructed building scenes in high-resolution remote sensing images is established. A vision-oriented scene description method is established, that is, the image is over-segmented through the superpixel method, and the homogeneous vision obtained by segmentation is used as a visual vocabulary unit. The spectral, texture and spatial information characteristics are described as visual words, and the hidden layer probabilistic semantic parsing model is improved to perform scene inference. By combining the imaging characteristics of remote sensing images and based on the scene content composition, a vision-oriented strategy is used to parse the distribution law of objects, improve the semantic carrying capacity of visual words, solve the semantic gap problem, and improve the mapping level of image semantics.

[0014] 1) High-resolution remote sensing image scene content expression based on segmentation vision: The image is over-segmented using the superpixel segmentation method, and the extracted segmentation vision is used as the semantic vocabulary. The segmented homogeneous area is taken as the object for feature extraction, and the semantic model is used for inference to convert the image content into collaborative distributed semantics;

[0015] Specifically, it includes: 1. Scene content inference based on target extraction: segment the image, use the image facets obtained by segmentation as the basic unit, and use color, texture, and spatial features to perform multi-feature calculations to achieve recognition and classification of visual targets; 2. Scene target expression based on visual features: describe the block vision obtained by over-segmentation from three aspects: layer value features, texture features, and spatial information features, and merge them to form a feature vector, where the layer features are obtained by calculating the spectral information of each band, and the spatial information features are calculated by calculating the distribution characteristics of the visual pixels in the image coordinates to achieve visual category information labeling; 3. Scene target visual acquisition based on multi-mode feature segmentation;

[0016] 2) Remote sensing image scene classification based on latent probabilistic semantic analysis: Accurately describe the scene content semantically, establish an effective relationship between high-level semantics and low-level features, adopt a semantic scene modeling method based on the middle layer, assume the abstract concepts hidden in the image in advance, and use the visual vocabulary method to represent the intermediate semantic features of the entity information in the image. The same scene has similar lexical concept distribution. The semantic attributes of the image are divided by the concept distribution of the intermediate semantic vocabulary. The scene target vocabulary expression method based on segmented vision is combined with the probability inference model to improve the level of semantic expression;

[0017] Specifically, it includes: 1. Hidden layer probability semantic parsing model; 2. Remote sensing image scene semantic parsing based on visual vocabulary; 3. Discriminative classification based on building recognition classifier.

[0018] Preferably, scene content inference based on target extraction: firstly, effectively extract and characterize the targets in the scene, then compress the image data through the hidden layer probabilistic semantic parsing method, transform the image from the spatial dimension to the focal space for abstract representation, reduce the feature dimension and information redundancy, and then use the classifier for effective classification, learn scene knowledge from limited samples, and automatically guide scene classification;

[0019] The segmentation strategy is used to process the image based on the overall characteristics of the pixel block, and then the image features are clustered within each object. The homogeneous area is obtained by over-segmenting the high spatial resolution image. The feature vector contains multi-dimensional semantic information by effectively representing the segmented area, and the different target visions are finely divided.

[0020] On the basis of precise visual description of the scene, the image is focused through latent probabilistic semantic analysis, and the distribution characteristics of various objects in the scene image are integrated to fully express the scene semantics. Based on the object composition of the scene, the spatial association, spatial configuration, and spatial interaction rules between target units are expressed by using analysis based on segmented visual primitives and semantic knowledge to abstract the semantic information content from the scene itself.

[0021] Preferably, the layer feature is the spectral feature of each band data. Compared with other visual features, the color feature has a small dependence on the size, direction and viewing angle of the image. The calculation method thereof includes:

[0022] average value The average value of all pixels that make up the visual representation of this image:

[0023]

[0024] In formula 1 represents the grayscale mean of layer L, n represents the number of pixels, C Li Represents the gray value of the i-th pixel, the same below;

[0025] Variance σ 2 L: The sum of the squares of all pixel values ​​and mean differences that make up the image vision:

[0026]

[0027] Brightness B: The number of image visual layers divided by the sum of the average values ​​of the layers containing spectral information, that is, the average value of the spectral average values ​​of the image visual:

[0028]

[0029] In formula 3, n L Represents the total number of bands in the image, Represents the mean value of vision in each band, the same below;

[0030] Contribution rate L : The average value of image vision at a certain layer divided by the sum of the average values ​​of all spectral layers, as shown in Formula 4:

[0031]

[0032] It reflects the ratio of the average spectrum of this layer to the total average of all layers.

[0033] Preferably, scene target visual acquisition based on multi-mode feature segmentation: clustering spatial feature points, drifting sample points along the gradient ascending direction until the translation vector is less than the minimum value, assuming {x i |i∈1,2,3…n} represents the d-dimensional Euclidean space R d The sample point set S is composed of any n points n At the sample point x, the kernel density function estimated by the kernel function K(x) and the window width h is used to calculate the contribution of each sample point in the neighborhood to the central point feature:

[0034]

[0035] where s k,d To satisfy the contour function k(), the inverse of the integral is 1 constant coefficient, n is the number of samples, h is the bandwidth size, and in image analysis, it is the size of the local neighborhood, x i is the i-th sample reference point, x is the center point of the kernel function, and through the density estimation of the kernel function, the average value of the local kernel function centered on each data point is used as the weighted local average of the estimated point in the window at the center of the estimated point, and the mean shift vector is calculated as:

[0036]

[0037] Weight coefficient v i Satisfy the constraints:

[0038]

[0039] The multimodal feature is calculated by calculating the multimodal feature vector of the center point of the window, and iterates step by step along the gradient ascent direction to a new position until the local maximum point approaches zero, the step size gradually decreases, and slowly approaches the convergence point.

[0040] Preferably, the hidden layer probability semantic parsing model: In the semantic parsing of image scenes, the classification of images is solved by establishing a probability model based on the input image’s computational features, that is, firstly by modeling the class conditional distribution p(x,c k ), x represents the input and output data, c k Represents the category, and then calculates the posterior probability p(c k,x) to predict the classification situation, transform the latent probability semantic analysis into a data co-occurrence probability calculation model, combine graph theory and probability calculation analysis, define concepts or random variables to represent visual attribute characteristics, and use graph models to represent the relationship between text, words, and focus.

[0041] Preferably, the model is constructed as follows: Assume that a database contains M texts, where d m represents the mth article. Assume that the set of words that make up all the texts is V, where w v represents the vth word, P(d m , w v ) represents word w v In d m The probability of occurrence in the text is introduced between the text and the word. A text contains multiple focuses, which obey the multinomial distribution. Assuming that there are K focuses, there are K implicit variables z in the model. k , converting the original text represented by vocabulary into a higher-level text represented by a focus vector;

[0042] The model defines the process of text generation focus and vocabulary as follows: Assume p(d m ) Generate a text d m , given the text generation condition, according to the probability P(z k |d m ) generates a focus z k , at a given focus z k Under the generated conditions, according to the probability P(w v |z k ) generates a visual word, where d m and w v Obtained from the text-word co-occurrence matrix, it is an observable variable. The premise of the model is that text and words are generated independently and have nothing to do with the latent focus, and words and latent focus are also generated independently and have nothing to do with the specific text;

[0043] Assume that given a text, the distribution P(d m |w v ), given a focus, infer w v The distribution of P(w v |z k ), P(z k |d m ) and P(w v |z k ) is the model factor to be estimated, and its co-likelihood function is obtained through training samples:

[0044]

[0045] get:

[0046]

[0047] Let w m,n represents the nth word in the mth text. According to the maximum likelihood estimation criterion, its log-likelihood function is

[0048] Taking the logarithm of both sides gives the log-likelihood function, and expanding the above formula gives:

[0049]

[0050] Where n(d m ,w v ), indicating the text d m Chinese word w v The number of occurrences, the latter item is related to z k Regardless, we get:

[0051]

[0052] Establish a latent probabilistic semantic parsing model.

[0053] Preferably, the maximization factor estimation is solved by using expectation maximization; the steps are as follows:

[0054] Step 1: Expectation step, randomly initialize factors to find the posterior probability of latent variables based on the current estimated factors;

[0055] Step 2: Based on the results of step 1, calculate the new factor value by maximizing the expectation of the logarithmic natural function;

[0056] Repeat the above two steps until the logarithmic natural function value converges;

[0057] In the first step, the probability P(z k |d m ,w v ), then assume that P(w v |z k ) and P(z k |d m ) Two factors are known;

[0058] In step 2, the lower limit L of the likelihood function is approximated by maximizing the expected value of the complete data likelihood:

[0059]

[0060] P(z k |d m ,w v ) can be obtained by calculating in step 1, P(wv |z k ) and P(z k |d m ) two factors satisfy two constraints, so the Lagrange multiplier method is used to transform the constrained optimization problem into an unconstrained optimization problem, and the objective function is v |z k ) and P(z k |d m ) and set the equation to 0 to obtain the iterative formula for the second step re-estimation:

[0061]

[0062]

[0063] The above formula is calculated repeatedly and the initial value is continuously corrected until the iteration converges. The maximum likelihood value of each iteration gradually increases to a certain convergence value.

[0064] Preferably, the semantic parsing of remote sensing image scenes based on visual vocabulary: using the over-segmentation features of high-resolution images, taking the segmented image vision as the basic feature vocabulary unit, on the basis of image segmentation, extracting the spectral, texture and shape features of each homogeneous region vision to form a feature vector set, which is used as a visual word to be included in the hidden layer probability semantic parsing model for classification. The specific steps are as follows:

[0065] Step 1: Extract image features: Perform mean shift segmentation on all images, extract features from over-segmented images, randomly select a portion of images as training images, use clustering to generate visual vocabulary for the feature vectors of these images, specify the number of words V, and obtain the word frequency matrix N(w v , d m ), where it represents the vocabulary w v In text d m Number of occurrences;

[0066] Step 2: Factor training: Factor estimation is performed on the training image data, and the text is specified to have K focal points. v |z k ) and P(z k |d m ), where P(w v |z k ) is used for subsequent image focus distribution vector calculation, P(z k |d m ) as a K-dimensional intermediate vector to represent each image for training the classifier;

[0067] Step 3: Test image: For the test image, the cluster center obtained in step 1 is used as the visual dictionary, and the similarity measurement is performed on each feature vector to obtain the text-word frequency matrix of the training image, and the focus distribution of the training image is calculated. According to formula 13, the focus probability vector P(z k |d test );

[0068] Step 4: Category judgment: Use the trained classifier to classify the test image, take the focus distribution vector of the training image as the training sample, and input the scene category label; input the focus vector of each test image obtained in step 3 into the building recognition classifier, classify each image, obtain the semantic label, obtain the classification confusion matrix, and evaluate the classification accuracy.

[0069] Preferably, based on the discriminant classification of the building recognition classifier: first find the optimal hyperplane, assuming that the sample X{x 1 ,x 2 ,…,x n}, the category label is y i ∈{1, -1}, n is the number of samples, R n For n-dimensional sample space, the classification problem is transformed into finding the optimal hyperplane y=wx+b, which is the plane with the largest distance from the two types of sample points and satisfies formula 13 for all sample points:

[0070]

[0071] Classification interval Need to meet Minimum, achieve the maximum classification interval, the optimal hyperplane problem is converted into an inequality constraint problem through Lagrange;

[0072] The choice of kernel function in building recognition and classification has a great impact on the classification performance. The introduction of kernel function makes the sample space linearly separable after the dimension is increased. The kernel function is calculated in the original sample space, and the classification problem in the high-dimensional space can be solved by linear method.

[0073] In high-resolution image classification, classification features are integrated with brightness, texture features and other information to improve classification results. The feature dimension reaches dozens or even hundreds of dimensions. Building recognition classification implicitly expresses data as a certain feature space through kernel functions, converting nonlinear classification problems into linear classification problems.

[0074] Construct a multi-class classifier and establish a building recognition classifier between any two categories of samples. Samples of k categories require the design of k(k-1) / 2 building recognition classifiers. When classifying an unknown sample, classify it between the classifiers trained for each of the two categories of samples. Perform a secret vote based on the classification results of each two-category classification. The number of votes for the category to which the classification belongs increases by one. The category with the most votes is the category of the unknown sample.

[0075] Compared with the prior art, the innovations and advantages of this application are:

[0076] (1) This application establishes a scene semantic classification method based on segmented vision, improves the latent probability semantic parsing model combined with the local feature descriptor method, provides effective spatial constraint information for remote sensing image scene classification, and improves the scene expression effect and classification accuracy. The first is a mapping method from low-level visual features to high-level semantic features: the high-level semantic expression of an image is inseparable from the underlying features. In view of the semantic gap problem and the spectral variability of ground object categories, a feature description method oriented to vision is established. Different features represent different features of the image, and their extraction methods and efficiencies vary. By selecting a combination of features with strong expressiveness and strong complementarity, the description accuracy of the segmented target is improved. The second is a high-resolution remote sensing image scene classification method based on latent probability semantic parsing: the probability model and the bag-of-words model are inspired by text processing. The vocabulary representation method using local features does not utilize spatial information and ignores the spatial relationship between visual words. Spatial information is particularly important in remote sensing images. This application uses the segmented image vision as a vocabulary unit, incorporates spatial information into the model, quantifies spatial information, and plays a role in scene recognition, thereby improving recognition accuracy and efficiency.

[0077] (2) In order to make up for the shortcomings of the method of directly using the underlying features for scene annotation, the method based on the middle-level semantic model has been introduced into the research of scene recognition of remote sensing images. However, the existing middle-level model classification methods are mostly based on local features of the image. The semantic gap between the underlying features and the high-level abstract information is still difficult to solve. When dealing with remote sensing images with complex structural information, there are inevitably problems of reduced interpretation accuracy and reduced classification rate. In view of the above problems, this application establishes a vision-oriented scene description method, that is, the image is over-segmented by the superpixel method, and the homogeneous vision obtained by segmentation is used as a visual vocabulary unit. By describing its spectral, texture and spatial information features as visual words, the hidden layer probabilistic semantic parsing model is used to perform scene inference. Compared with the method based on local features, this application combines the imaging features of remote sensing images, based on the scene content composition, and adopts a vision-oriented strategy to analyze the distribution law of objects, improve the semantic carrying capacity of visual words, solve the problem of semantic gap, and improve the mapping level of image semantics. According to the experimental results, the classification accuracy of the method proposed in this application is relatively high, and good classification accuracy is achieved with low feature dimension and operation efficiency, and it has a good description effect on typical artificial scenes.

[0078] (3) The accuracy of remote sensing semantic information extraction in the existing technology is not high. Due to the complex features of remote sensing images, local features lead to serious semantic gap problems. The use of multi-feature methods will face the problems of large data volume and time-consuming algorithms. In response to the above problems, this application establishes a vision-oriented hidden layer probabilistic semantic parsing scene classification method, adds spatial information constraints to the features of scene expression, carries more semantic information with lower feature dimensions, improves the expression power of visual vocabulary, and improves the accuracy of scene classification. First, the image performance characteristics of typical artificial scenes are analyzed by combining the data features of high spatial resolution remote sensing images, and a lower feature dimension is used to describe the target visual information; second, combined with the characteristics of remote sensing images, the hidden layer probabilistic semantic parsing model is improved, and the strategy of segmenting the most characteristic vocabulary of vision is adopted to express the visual features of the over-segmented image, and the rich image spatial information and geometric structure information brought by high spatial resolution are used to make up for the limited description power of local feature points; the innovation of this application also lies in proposing a vision-oriented scene vocabulary expression method, combining the vision-oriented classification method with the focus model, and using low-dimensional feature description to achieve better classification results. Compared with the local feature based method, semantic vision is added to the semantic mapping level to increase the semantic carrying capacity of visual vocabulary.

[0079] Figure determination

[0080] Figure 1 It is a flow chart of semantic parsing of remote sensing image scenes based on visual vocabulary.

[0081] Figure 2It is a schematic diagram of the maximum likelihood value calculation in the factor estimation process.

[0082] Figure 3 This is the confusion matrix diagram of scene classification when the number of words K = 300 and the number of focuses Z = 60.

[0083] Figure 4 This is a diagram of the classification accuracy of each category when the number of words is 500 and the number of focuses is 60.

[0084] Figure 5 This is a schematic diagram of some of the misclassified scenes in the experimental results.

[0085] Figure 6 It is a schematic diagram of the change of overall classification accuracy with the number of training samples.

[0086] Figure 7 This is a schematic diagram of the change in overall classification accuracy for different bandwidths.

[0087] Figure 8 This is a schematic diagram of the algorithm time consumption for each step of the experimental scene classification. DETAILED DESCRIPTION

[0088] The following, in conjunction with the accompanying drawings, further describes the technical solution of the intelligent classification method for artificially constructed building scenes in high-resolution remote sensing images provided by the present application, so that technicians in the field can better understand the present application and implement it.

[0089] In order to make up for the shortcomings of the method of directly using the underlying features for scene annotation, the method based on the middle-level semantic model has been introduced into the research of scene recognition of remote sensing images. However, the existing middle-level model classification methods are mostly based on local features of the image, and the semantic gap between the underlying features and the high-level abstract information is still difficult to solve. In dealing with remote sensing images with complex structural information, there are inevitably problems of reduced interpretation accuracy and reduced classification rate. In response to the above problems, this application establishes a vision-oriented scene description method, that is, the image is over-segmented by the superpixel method, and the homogeneous vision obtained by segmentation is used as a visual vocabulary unit. By describing its spectral, texture and spatial information features as visual words, the hidden layer probabilistic semantic parsing model is used to perform scene inference. Compared with the method based on local features, this application combines the imaging features of remote sensing images, based on the scene content composition, and adopts a vision-oriented strategy to parse the distribution law of objects, improve the semantic carrying capacity of visual words, solve the problem of semantic gap, and improve the mapping level of image semantics.

[0090] To verify the effectiveness of this method, this application uses the USGS high-resolution remote sensing scene image dataset to conduct classification experiments, including typical artificial scenes, and semantically describes the mean shift segmentation vision and combines the hidden layer probabilistic semantic model to classify the scene. The visual bag of words model is used in combination with the scale-invariant feature classification experiment for comparison. According to the analysis of the experimental results, the classification accuracy of the method proposed in this application is relatively high, and good classification accuracy is achieved with low feature dimension and computational efficiency, and it has a good description effect on typical artificial scenes.

[0091] 1. Scene content expression of high-resolution remote sensing images based on segmentation vision

[0092] The superpixel segmentation method is used to over-segment the image, and the extracted segmented vision is used as the semantic vocabulary. The segmented homogeneous area is taken as the object for feature extraction, and the semantic model is used for inference to convert the image content into collaborative distributed semantics.

[0093] 1. Scene content estimation based on target extraction

[0094] The image is segmented, and the image pixels obtained by segmentation are used as basic units. Multi-feature calculations are performed using color, texture, and spatial features to achieve recognition and classification of visual targets.

[0095] First, the targets in the scene are effectively extracted and characterized. Then, the image data is compressed through the latent probabilistic semantic parsing method, and the image is transformed from the spatial dimension to the focal space for abstract representation, reducing the feature dimension and information redundancy, avoiding the huge amount of calculation caused by excessive dimensionality, and then using a classifier for effective classification, learning scene knowledge from limited samples, and automatically guiding scene classification without the need for human knowledge intervention.

[0096] The segmentation strategy is used to process images based on the overall characteristics of pixel blocks, avoiding the drawbacks of processing in pixels. Spatial semantic information is better utilized. Then, image feature clustering is performed within each object. By over-segmenting high spatial resolution images, homogeneous regions are obtained. By effectively representing the features of the segmented regions, the feature vector contains multi-dimensional semantic information, and finely divides different target visions.

[0097] On the basis of precise visual description of the scene, the image is focused through latent probabilistic semantic analysis, and the distribution characteristics of various objects in the scene image are integrated to fully express the scene semantics. Based on the object composition of the scene, the spatial association, spatial configuration, and spatial interaction rules between target units are expressed by using analysis based on segmented visual primitives and semantic knowledge to abstract the semantic information content from the scene itself.

[0098] 2. Scene target expression based on visual features

[0099] The visual image obtained after segmentation is approximately corresponding to the real ground entity. The homogeneous area after segmentation contains a variety of spatial information. Feature selection, analysis and extraction are performed on the segmented vision as the feature basis for subsequent classification.

[0100] The visual block obtained by over-segmentation is described from three aspects: layer value features, texture features and spatial information features, and they are merged to form a feature vector. The layer features are obtained by calculating the spectral information of each band, and the spatial information features are obtained by calculating the distribution characteristics of the visual pixels in the image coordinates to achieve visual category information labeling.

[0101] 1. Layer characteristics

[0102] Layer features are the spectral features of each band of data. Compared with other visual features, color features are less dependent on the size, direction and viewing angle of the image. The calculation methods include:

[0103] average value The average value of all pixels that make up the visual representation of this image:

[0104]

[0105] In formula 1 represents the grayscale mean of layer L, n represents the number of pixels, C Li Represents the gray value of the i-th pixel, the same below;

[0106] Variance σ 2 L: The sum of the squares of all pixel values ​​and mean differences that make up the image vision:

[0107]

[0108] Brightness B: The number of image visual layers divided by the sum of the average values ​​of the layers containing spectral information, that is, the average value of the spectral average values ​​of the image visual:

[0109]

[0110] In formula 3, n L Represents the total number of bands in the image, Represents the mean value of vision in each band, the same below;

[0111] Contribution rate L : The average value of image vision at a certain layer divided by the sum of the average values ​​of all spectral layers, as shown in Formula 4:

[0112]

[0113] It reflects the ratio of the average spectrum of this layer to the sum of the average spectrum of all layers.

[0114] 2. Texture features

[0115] The pixel distribution pattern is expressed qualitatively and quantitatively, and the second-order combination conditional probability density of the image is estimated. In the θ direction, a pair of pixels separated by a distance of d pixels have the probability of appearing in grayscale layers i and j respectively. The grayscale is compressed and layered to express the texture characteristics in the image area.

[0116] 3. Visual acquisition of scene targets based on multi-modal feature segmentation

[0117] 1. High-resolution image segmentation method

[0118] The mean shift method is used to segment the scene image, and then the scene target expression based on visual features is used for feature description. Image segmentation is the key point connecting low-level image processing and mid-to-high-level image analysis.

[0119] The sampling interval of high-resolution image data is reduced, which makes high-resolution remote sensing present new features different from multispectral data, resulting in the widespread problem of increased intra-class differences and reduced inter-class differences of objects. The existing technical algorithms often lead to serious over-segmentation of high-resolution remote sensing image data, fragmented image segmentation spots, and large noise images. Therefore, it is necessary to use other information such as texture to improve the segmentation quality. The segmentation results of the existing technology are quite different from the semantic vision perceived by people, and the segmentation accuracy and stability need to be improved.

[0120] 2. Multimodal feature segmentation method

[0121] Cluster the spatial feature points, and drift the sample points along the gradient ascending direction until the translation vector is less than the minimum value. Assume {x i |i∈1,2,3…n} represents the d-dimensional Euclidean space R d The sample point set S is composed of any n points n At the sample point x, the kernel density function estimated by the kernel function K(x) and the window width h is used to calculate the contribution of each sample point in the neighborhood to the central point feature:

[0122]

[0123] where s k,d To satisfy the contour function k(), the inverse of the integral is 1 constant coefficient, n is the number of samples, h is the bandwidth size, and in image analysis, it is the size of the local neighborhood, x i is the i-th sample reference point, x is the center point of the kernel function, and through the density estimation of the kernel function, the average value of the local kernel function centered on each data point is used as the weighted local average of the estimated point in the window at the center of the estimated point, and the mean shift vector is calculated as:

[0124]

[0125] Weight coefficient v i Satisfy the constraints:

[0126]

[0127] The multimodal feature is calculated by calculating the multimodal feature vector of the center point of the window, and iterates step by step along the gradient ascent direction to a new position until the local maximum point approaches zero, the step size gradually decreases, and slowly approaches the convergence point.

[0128] 2. Remote Sensing Image Scene Classification Based on Hidden Layer Probabilistic Semantic Parsing

[0129] In order to overcome the limitations of scene classification methods that directly use low-level features and make an accurate semantic description of scene content, it is necessary to establish an effective relationship between high-level semantics and low-level features. A semantic scene modeling method based on the middle layer is adopted. By assuming the abstract concepts implicit in the image in advance, a visual vocabulary method is used to represent the intermediate semantic features of the entity information in the image. The same type of scenes have similar vocabulary concept distributions, and there are certain differences between different scenes. The semantic attributes of the image are divided through the concept distribution of intermediate semantic vocabulary. The scene target vocabulary expression method based on segmentation vision is combined with the probability inference model to improve the level of semantic expression.

[0130] (I) Hidden layer probabilistic semantic parsing model

[0131] In image scene semantic analysis, the classification of images is solved by building a probability model based on the input image’s computational features. That is, firstly, we build a class conditional distribution p(x,c k ), x represents the input and output data, c k Represents the category, and then calculates the posterior probability p(c k ,x) to predict the classification situation, transform the latent probability semantic analysis into a data co-occurrence probability calculation model, combine graph theory and probability calculation analysis, define concepts or random variables to represent visual attribute characteristics, and use graph models to represent the relationship between text, words, and focus.

[0132] 1. Model construction

[0133] Suppose a database contains M texts, where d m represents the mth article. Assume that the set of words that make up all the texts is V, where w v represents the vth word, P(d m , w v ) represents word w v In d mThe probability of occurrence in the text is introduced between the text and the word. A text contains multiple focuses, which obey the multinomial distribution. Assuming that there are K focuses, there are K implicit variables z in the model. k , converting the original text represented by vocabulary into a higher-level text represented by a focus vector;

[0134] The model defines the process of text generation focus and vocabulary as follows: Assume p(d m ) Generate a text d m , given the text generation condition, according to the probability P(z k |d m ) generates a focus z k , at a given focus z k Under the generated conditions, according to the probability P(w v |z k ) generates a visual word, where d m and w v Obtained from the text-word co-occurrence matrix, it is an observable variable. The premise of the model is that text and words are generated independently and have nothing to do with the latent focus, and words and latent focus are also generated independently and have nothing to do with the specific text;

[0135] Assume that given a text, the distribution P(d m |w v ), given a focus, infer w v The distribution of P(w v |z k ), P(z k |d m ) and P(w v |z k ) is the model factor to be estimated, and its co-likelihood function is obtained through training samples:

[0136]

[0137] get:

[0138]

[0139] Let w m,n represents the nth word in the mth text. According to the maximum likelihood estimation criterion, its log-likelihood function is

[0140] Taking the logarithm of both sides gives the log-likelihood function, and expanding the above formula gives:

[0141]

[0142] Where n(d m ,w v), indicating the text d m Chinese word w v The number of occurrences, the latter item is related to z k Regardless, we get:

[0143]

[0144] Establish a latent probabilistic semantic parsing model.

[0145] 2. Maximizing Factor Estimation

[0146] The solution is solved by expectation maximization; the steps are as follows:

[0147] Step 1: Expectation step, randomly initialize factors to find the posterior probability of latent variables based on the current estimated factors;

[0148] Step 2: Based on the results of step 1, calculate the new factor value by maximizing the expectation of the logarithmic natural function;

[0149] Repeat the above two steps until the logarithmic natural function value converges;

[0150] In the first step, the probability P(z k |d m ,w v ), then assume that P(w v |z k ) and P(z k |d m ) Two factors are known;

[0151] In step 2, the lower limit L of the likelihood function is approximated by maximizing the expected value of the complete data likelihood:

[0152]

[0153] P(z k |d m ,w v ) can be obtained by calculating in step 1, P(w v |z k ) and P(z k |d m ) two factors satisfy two constraints, so the Lagrange multiplier method is used to transform the constrained optimization problem into an unconstrained optimization problem, and the objective function is v |z k ) and P(z k |d m ) and set the equation to 0 to obtain the iterative formula for the second step re-estimation:

[0154]

[0155]

[0156] The above formula is calculated repeatedly and the initial value is continuously corrected until the iteration converges. The maximum likelihood value of each iteration gradually increases to a certain convergence value.

[0157] 2. Semantic parsing of remote sensing image scenes based on visual vocabulary

[0158] Using the over-segmentation features of high-resolution images, the segmented image vision is used as the basic feature vocabulary unit. On the basis of image segmentation, the spectral, texture and shape features of each homogeneous region vision are extracted to form a feature vector set, which is used as a visual word to be included in the hidden layer probabilistic semantic parsing model for classification. The flow chart is as follows Figure 1 The specific steps are as follows:

[0159] Step 1: Extract image features: Perform mean shift segmentation on all images, extract features from over-segmented images, randomly select a portion of images as training images, use clustering to generate visual vocabulary for the feature vectors of these images, specify the number of words V, and obtain the word frequency matrix N(w v , d m ), where it represents the vocabulary w v In text d m Number of occurrences;

[0160] Step 2: Factor training: Factor estimation is performed on the training image data, and the text is specified to have K focal points. v |z k ) and P(z k |d m ), where P(w v |z k ) is used for subsequent image focus distribution vector calculation, P(z k |d m ) as a K-dimensional intermediate vector to represent each image for training the classifier;

[0161] Step 3: Test image: For the test image, the cluster center obtained in step 1 is used as the visual dictionary, and the similarity measurement is performed on each feature vector to obtain the text-word frequency matrix of the training image, and the focus distribution of the training image is calculated. According to formula 13, the focus probability vector P(z k |d test ).

[0162] Step 4: Category judgment: Use the trained classifier to classify the test image, use the focus distribution vector of the training image as the training sample, and input the scene category label. Input the focus vector of each test image obtained in step 3 into the building recognition classifier, classify each image, obtain the semantic label, obtain the classification confusion matrix, and evaluate the classification accuracy.

[0163] (III) Discriminant classification based on building recognition classifier

[0164] First, find the optimal hyperplane, assuming that the sample X{x 1 ,x 2 ,…,x n}, the category label is y i ∈{1, -1}, n is the number of samples, R n For n-dimensional sample space, the classification problem is transformed into finding the optimal hyperplane y=wx+b, which is the plane with the largest distance from the two types of sample points and satisfies formula 13 for all sample points:

[0165]

[0166] Classification interval Need to meet Minimum, achieve the maximum classification interval, the optimal hyperplane problem is converted into an inequality constraint problem through Lagrange;

[0167] The choice of kernel function in building recognition and classification has a great impact on the classification performance. The introduction of kernel function makes the sample space linearly separable after the dimension is increased. The kernel function is calculated in the original sample space, and the classification problem in the high-dimensional space can be solved by linear method.

[0168] In the high-resolution image classification problem, the classification features are no longer limited to the pixel grayscale features of low- and medium-resolution remote sensing images, but the features of brightness, texture and other information are integrated to improve the classification effect, and the feature dimension reaches dozens or even hundreds of dimensions. Building recognition classification implicitly expresses the data as a certain feature space through the kernel function, converting the nonlinear classification problem into a linear classification problem.

[0169] Construct a multi-class classifier and establish a building recognition classifier between any two categories of samples. Samples of k categories require the design of k(k-1) / 2 building recognition classifiers. When classifying an unknown sample, classify it between the classifiers trained for each of the two categories of samples. Perform a secret vote based on the classification results of each two-category classification. The number of votes for the category to which the classification belongs increases by one. The category with the most votes is the category of the unknown sample.

[0170] 3. Experimental results and analysis

[0171] This application uses remote sensing images for scene classification experiments for verification. The comparative experiment is the BOVW method, which generates visual word bags by extracting SIFT features of the image and clustering them, then calculates the frequency histogram of the words and uses building recognition for classification.

[0172] (I) Experimental data

[0173] The experimental dataset contains a rich variety of ground objects and structures. The image specifications are RGB color images with a spatial resolution of about 0.3m. The image content includes 21 scene images, including 16 artificial scenes, covering typical scene contents such as parking lots, ports, airports, residential areas, stadiums, and intersections. Each scene contains 100 images, totaling 1,600 images. The size of each image is 256×256 pixels.

[0174] 2. Evaluation Method

[0175] (1) Mixed Silk Matrix

[0176] Classify each test sample, build a confusion matrix based on the actual category label and classification category label of the sample, and perform consistency check on each test image sample and actual data;

[0177]

[0178] In formula (14), n represents the type of ground feature, m ij It represents the number of samples that are actually classified into class j for class i. The diagonal (i=j) represents the number of samples that are correctly classified. The confusion matrix intuitively reflects the number of samples in each class that are correctly classified, as well as the number of misclassifications and omissions.

[0179] (2) Overall classification accuracy

[0180] It indicates the percentage of the total number of correctly classified samples in the classification to all test samples. The calculation method is:

[0181]

[0182] Where n represents the number of sample types, r i represents the number of test samples correctly classified in the i-th category, s i Represents the total number of test samples of the i-th category.

[0183] (III) Analysis of experimental results

[0184] After the image is segmented, the feature extraction method for vision is used for feature representation. The layer value features, texture features and spatial features of the segmented vision are extracted for integrated representation. Visual words are formed by clustering, and the factors are estimated by the hidden layer probability semantic parsing model. During the test, the test samples are converted into focus distribution vectors by the hidden layer probability semantic parsing model and the existing estimation factors, and then the samples are discriminated and classified. The control experiment extracts the image SIFT features as visual words, and calculates the histogram distribution after clustering. Both groups of experiments use the building recognition classifier method for classification. The kernel function used in the building recognition classification is the radial basis kernel function, and the default factor is used. The number of clustered words in the control group experiment is 1200.

[0185] 1. Classification effect analysis and evaluation

[0186] A good classification accuracy can be achieved when the number of words is selected as 300 and the number of focus is 60. Before classification, factor estimation is required based on sample data to obtain the co-occurrence probability relationship between focus words and the focus distribution of samples for testing and classification. In this application experiment, 50 images of each type are selected for training, totaling 800 samples.

[0187] like Figure 2 , the maximum likelihood value calculation status in the factor estimation process can effectively estimate the factor in the case of missing data. During the iteration process, the maximum likelihood value will continue to increase and gradually converge to a certain value. v |z k ) and P(z k |d m ) has a good convergence speed. It can be seen from the curve in the figure that after 30 iterations, the maximum likelihood estimation value basically converges.

[0188] Figure 3 The confusion matrix obtained by classification with 50 training samples for each scene is given. The numbers in the first row of the figure correspond to the scene categories in the first column.

[0189] like Figure 3 The confusion matrix and Figure 4 The classification results of various scenes are shown, and the overall classification accuracy is 80.38%. The classification accuracy of golf courses, ports, parking lots and runways is relatively high, at 98.0%, 96.0%, 94.0% and 94.0% respectively. In the classification results of other groups of different factors, the classification accuracy of these types of targets is also higher than that of other scenes.

[0190] like Figure 5Some of the scene images that were misclassified are shown. There are two classification errors in this application: one is that there are errors in the image segmentation process. When there are many scene images, there are also many types of ground objects. There is an optimal segmentation scale for a certain target, but the same scale is often unsatisfactory when segmenting other ground objects, and there is spectral similarity between adjacent targets in some images, which will cause merging. Figure 5 (a), (b), and (c) are examples of rare cases where it is not possible to effectively separate targets such as airplanes, oil depots, and tennis courts from background areas during segmentation. If typical targets in the image cannot be effectively segmented and extracted, accurate feature descriptions cannot be performed, and airports and oil depots are mistakenly classified as buildings, and tennis courts are mistakenly classified as sparse residential areas. Second, scene images composed of similar objects are very similar, and similar focus distributions will appear during training and testing. Figure 5 The images (d), (e), and (f) are different categories of scenes, belonging to mid-level residential areas, parking residential areas, and intersections. These three scenes have great inter-class similarity, and there is also great variation and diversity between the same categories. After segmentation, they have similar word frequency representations, which leads to classification errors.

[0191] 2. Overall classification accuracy changes with the number of training samples

[0192] The classic BOVW method is used as an experimental comparison. This method is a typical method in remote sensing image scene classification. In order to compare the effectiveness of the scene classification proposed in this application, the scene description method is experimentally compared with the BOVW method for analysis and evaluation.

[0193] like Figure 6 As shown, in order to reduce the impact of randomness of sample selection on the experimental results, training samples were randomly selected from each type of scene in the experiment, and the experiment was repeated 5 times, and the overall classification accuracy was averaged. In the experiment, the number of image samples of each type of scene was selected as 40, 50, 60, and 70, respectively, and the corresponding number of overall training samples was 640, 800, 960, and 1120, respectively. The method of this application and the comparative method were used for experiments respectively. The overall classification accuracy achieved by the method of this application was 77.65%, 80.51%, 80.74%, and 81.44%, respectively, and the classification accuracy of the control group experiment was 74.03%, 78.39%, 78.16%, and 79.78%, respectively. With the increase in the number of samples, the overall classification accuracy has been improved. The average classification accuracy of the method of this application is 2.49% higher than that of the BOVW method, and the overall classification accuracy is better than that of the control group experiment, which proves the effectiveness of the method proposed in this application.

[0194] 3. Impact of different segmentation factors on scene classification accuracy

[0195] like Figure 7 As shown in the figure, different bandwidths have different effects on the overall classification accuracy. Five groups of different bandwidth values ​​are set for experiments, and the corresponding classification accuracies are 71.03%, 80.38%, 78.60%, 78.16%, and 69.78%, respectively. When the spatial bandwidth is 7, the visual integrity of the object is difficult to guarantee, the fitting effect of the segmented edge and the actual edge is poor, and the segmented area presents more irregular polygons, resulting in large errors in the description of spatial information. A higher classification accuracy can be achieved when the bandwidth is 10. When the spatial bandwidth reaches 21, there are a large number of regional merging phenomena for different ground objects, which not only leads to a decrease in the accuracy of the spectral feature solution, but also leads to errors in the description of spatial information. Due to the diversity of scene image targets, the optimal segmentation scales of different ground objects are different, and it is necessary to manually select a suitable spatial bandwidth for segmentation.

[0196] 4. Analysis of algorithm efficiency

[0197] The experiment and the control experiment both include two steps: sample training and testing. The training includes feature extraction and vocabulary clustering. In the method proposed in this application, feature extraction also includes multi-modal feature segmentation. The test process is the feature classification of the sample and the generation of the vocabulary frequency matrix. The discriminant classification includes the training of the building recognition classifier and the classification of the test image. Figure 8 The time consumption of each algorithm step in the experiment is listed in seconds, where the feature extraction and sample testing time refers to the processing time of each sample.

[0198] from Figure 8 It can be seen that the feature extraction of the method of the present application includes the link of image segmentation, which is slightly more time-consuming than SIFT feature extraction, but it greatly reduces the time consumption in the subsequent clustering and test sample vocabulary classification, and the time consumption in the subsequent sample testing and classification is also significantly reduced. Because with the increase of feature dimension and the number of clustering words, the time consumption of the algorithm will increase rapidly. In the data set used in this application, there are as many as 16 kinds of scene images. When using 128-dimensional SIFT features for feature space clustering, the number of words needs to be set to more than 1000 to achieve a good classification accuracy. The features used in this application are compact and concise, which greatly reduces the time consumption and has an efficiency advantage when processing a large amount of multi-class high-resolution image data.

[0199] The method proposed in this application can obtain better classification accuracy with fewer feature dimensions, does not require supervised learning of various ground objects, and has a good degree of automation. Compared with the method based on local features, the feature expression method oriented to vision greatly reduces the clustering time of samples and improves the efficiency of the algorithm.

Claims

1. An intelligent classification method for artificial building scenes in high-resolution remote sensing images, characterized in that: Establish a vision-oriented scene description method, that is, to over-segment the image through the superpixel method, use the homogeneous vision obtained by segmentation as the visual vocabulary unit, describe its spectral, texture and spatial information characteristics as visual words, improve the hidden layer probability semantic parsing model for scene inference, combine the imaging characteristics of remote sensing images, based on the scene content composition, adopt a vision-oriented strategy to parse the distribution law of objects, improve the semantic carrying capacity of visual words, solve the semantic gap problem, and improve the mapping level of image semantics; 1) High-resolution remote sensing image scene content expression based on segmentation vision: The image is over-segmented using the superpixel segmentation method, and the extracted segmentation vision is used as the semantic vocabulary. The segmented homogeneous area is taken as the object for feature extraction, and the semantic model is used for inference to convert the image content into collaborative distributed semantics; Specifically, it includes: First, scene content inference based on target extraction: segment the image, use the image facets obtained by segmentation as the basic unit, and use color, texture, and spatial features to perform multi-feature calculations to achieve recognition and classification of visual targets; second, scene target expression based on visual features: describe the block vision obtained by over-segmentation from three aspects: layer value features, texture features, and spatial information features, and merge them to form a feature vector, where the layer features are obtained by calculating the spectral information of each band, and the spatial information features are calculated by calculating the distribution characteristics of the visual pixels in the image coordinates to achieve visual category information labeling; third, scene target visual acquisition based on multi-mode feature segmentation: cluster the spatial feature points, and drift the sample points along the gradient ascent direction until the translation vector is less than the minimum value. Assume that {x i |i∈1,2,3…n} represents the d-dimensional Euclidean space R d The sample point set S is composed of any n points n At the sample point x, the kernel density function estimated by the kernel function K(x) and the window width h is used to calculate the contribution of each sample point in the neighborhood to the central point feature: where s k,d To satisfy the contour function k(), the inverse of the integral is 1 constant coefficient, n is the number of samples, h is the bandwidth size, and in image analysis, it is the size of the local neighborhood, x i is the i-th sample reference point, x is the center point of the kernel function, and through the density estimation of the kernel function, the average value of the local kernel function centered on each data point is used as the weighted local average of the estimated point in the window at the center of the estimated point, and the mean shift vector is calculated as: Weight coefficient v i Satisfy the constraints: The multi-mode feature is calculated by calculating the multi-mode feature vector of the center point of the window, and iterates step by step along the gradient ascending direction to the new position until the local maximum point approaches zero, the step size gradually decreases, and slowly approaches the convergence point; 2) Remote sensing image scene classification based on latent probabilistic semantic analysis: Accurately describe the scene content semantically, establish an effective relationship between high-level semantics and low-level features, adopt a semantic scene modeling method based on the middle layer, assume the abstract concepts hidden in the image in advance, and use the visual vocabulary method to represent the intermediate semantic features of the entity information in the image. The same scene has similar lexical concept distribution. The semantic attributes of the image are divided by the concept distribution of the intermediate semantic vocabulary. The scene target vocabulary expression method based on segmented vision is combined with the probability inference model to improve the level of semantic expression; Specifically, it includes: first, the latent probability semantic parsing model; second, the semantic parsing of remote sensing image scenes based on visual vocabulary; third, the discriminant classification based on the building recognition classifier; Hidden layer probability semantic parsing model: In the semantic parsing of image scenes, the classification of images is solved by building a probability model based on the input image’s computational features. That is, firstly, the class conditional distribution p(x,c k ), x represents the input and output data, c k Represents the category, and then calculates the posterior probability p(c k ,x) to predict the classification situation, transform the latent probability semantic analysis into a data co-occurrence probability calculation model, combine graph theory with probability calculation analysis, define concepts or random variables to represent visual attribute features, and use graph models to represent the relationship between text, words, and focus; Semantic parsing of remote sensing image scenes based on visual vocabulary: Using the over-segmentation features of high-resolution images, the segmented image vision is used as the basic feature vocabulary unit. On the basis of image segmentation, the spectral, texture and shape features of each homogeneous region vision are extracted to form a feature vector set, which is then included in the hidden layer probabilistic semantic parsing model as a visual word for classification. The specific steps are as follows: Step 1: Extract image features: Perform mean shift segmentation on all images, extract features from over-segmented images, randomly select a portion of images as training images, use clustering to generate visual vocabulary for the feature vectors of these images, specify the number of words V, and obtain the word frequency matrix N(w v , d m ), where it represents the vocabulary w v In text d m Number of occurrences; Step 2: Factor training: Factor estimation is performed on the training image data, and the text is specified to have K focal points. v |z k ) and P(z k |d m ), where P(w v |z k ) is used for subsequent image focus distribution vector calculation, P(z k |d m ) as a K-dimensional intermediate vector to represent each image for training the classifier; Step 3: Test image: For the test image, the cluster center obtained in step 1 is used as the visual dictionary, and the similarity measurement is performed on each feature vector to obtain the text-word frequency matrix of the training image, and the focus distribution of the training image is calculated. According to formula 13, the focus probability vector P(z k |d test ); Step 4: Category judgment: Use the trained classifier to classify the test image, use the focus distribution vector of the training image as the training sample, and input the scene category label; input the focus vector of each test image obtained in step 3 into the building recognition classifier, classify each image, obtain the semantic label, obtain the classification confusion matrix, and evaluate the classification accuracy; Discriminant classification based on building recognition classifier: First, find the optimal hyperplane, assuming that the sample X{x1,x2,…,x n }, the category label is y i ∈{1, -1}, n is the number of samples, R n For n-dimensional sample space, the classification problem is transformed into finding the optimal hyperplane y=wx+b, which is the plane with the largest distance from the two types of sample points and satisfies formula 13 for all sample points: Classification interval Need to meet Minimum, achieve the maximum classification interval, the optimal hyperplane problem is converted into an inequality constraint problem through Lagrange; The choice of kernel function in building recognition and classification has a great impact on the classification performance. The introduction of kernel function makes the sample space linearly separable after the dimension is increased. The kernel function is calculated in the original sample space, and the classification problem in the high-dimensional space can be solved by linear method. In high-resolution image classification, classification features are integrated with brightness, texture features and other information to improve classification results. The feature dimension reaches dozens or even hundreds of dimensions. Building recognition classification implicitly expresses data as a certain feature space through kernel functions, converting nonlinear classification problems into linear classification problems. Construct a multi-class classifier and establish a building recognition classifier between any two categories of samples. Samples of k categories require the design of k(k-1) / 2 building recognition classifiers. When classifying an unknown sample, classify it between the classifiers trained for each of the two categories of samples. Perform a secret vote based on the classification results of each two-category classification. The number of votes for the category to which the classification belongs increases by one. The category with the most votes is the category of the unknown sample.

2. According to the method for intelligent classification of artificial building scenes in high-resolution remote sensing images according to claim 1, it is characterized in that: Scene content inference based on target extraction: First, effectively extract and characterize the targets in the scene, then compress the image data through the hidden layer probabilistic semantic parsing method, transform the image from the spatial dimension to the focal space for abstract representation, reduce the feature dimension and information redundancy, and then use the classifier for effective classification, learn scene knowledge from limited samples, and automatically guide scene classification; The segmentation strategy is used to process the image based on the overall characteristics of the pixel block, and then the image features are clustered within each object. The homogeneous area is obtained by over-segmenting the high spatial resolution image. The feature vector contains multi-dimensional semantic information by effectively representing the segmented area, and the different target visions are finely divided. On the basis of precise visual description of the scene, the image is focused through latent probabilistic semantic analysis, and the distribution characteristics of various objects in the scene image are integrated to fully express the scene semantics. Based on the object composition of the scene, the spatial association, spatial configuration, and spatial interaction rules between target units are expressed by using analysis based on segmented visual primitives and semantic knowledge to abstract the semantic information content from the scene itself.

3. According to the method for intelligent classification of artificial building scenes in high-resolution remote sensing images as described in claim 1, it is characterized by: Layer features are the spectral features of each band of data. Compared with other visual features, color features are less dependent on the size, direction and viewing angle of the image. The calculation methods include: average value The average value of all pixels that make up the visual representation of this image: In formula 1 represents the grayscale mean of layer L, n represents the number of pixels, C Li Represents the gray value of the i-th pixel, the same below; Variance σ 2 L: The sum of the squares of all pixel values ​​and mean differences that make up the image vision: Brightness B: The number of image visual layers divided by the sum of the average values ​​of the layers containing spectral information, that is, the average value of the spectral average values ​​of the image visual: In formula 3, n L Represents the total number of bands in the image, Represents the mean value of vision in each band, the same below; Contribution rate L : The average value of image vision at a certain layer divided by the sum of the average values ​​of all spectral layers, as shown in Formula 4: It reflects the ratio of the average spectrum of this layer to the sum of the average spectrum of all layers.

4. The intelligent classification method for artificial building scenes in high-resolution remote sensing images according to claim 1 is characterized in that: Model construction: Assume a database contains M texts, where d m represents the mth article. Assume that the set of words that make up all the texts is V, where w v represents the vth word, P(d m , w v ) represents word w v In d m The probability of occurrence in the text is introduced between the text and the word. A text contains multiple focuses, which obey the multinomial distribution. Assuming that there are K focuses, there are K implicit variables z in the model. k , converting the original text represented by vocabulary into a higher-level text represented by a focus vector; The model defines the process of text generation focus and vocabulary as follows: Assume p(d m ) Generate a text d m , given the text generation condition, according to the probability P(z k |d m ) generates a focus z k , at a given focus z k Under the generated conditions, according to the probability P(w v |z k ) generates a visual word, where d m and w v Obtained from the text-word co-occurrence matrix, it is an observable variable. The premise of the model is that text and words are generated independently and have nothing to do with the latent focus, and words and latent focus are also generated independently and have nothing to do with the specific text; Assume that given a text, the distribution of focus z can be inferred P(d m |w v ), given a focus, infer w v The distribution of P(w v |z k ), P(z k |d m ) and P(w v |z k ) is the model factor to be estimated, and its co-likelihood function is obtained through training samples: get: Let w m,n represents the nth word in the mth text. According to the maximum likelihood estimation criterion, its log-likelihood function is Taking the logarithm of both sides gives the log-likelihood function, and expanding the above formula gives: Where n(d m ,w v ), indicating the text d m Chinese word w v The number of occurrences, the latter item is related to z k Regardless, we get: Establish a latent probabilistic semantic parsing model.

5. The intelligent classification method for artificial building scenes in high-resolution remote sensing images according to claim 1 is characterized in that: Maximization factor estimation: Expectation maximization is used for solution; the steps are as follows: Step 1: Expectation step, randomly initialize factors to find the posterior probability of latent variables based on the current estimated factors; Step 2: Based on the results of step 1, calculate the new factor value by maximizing the expectation of the logarithmic natural function; Repeat the above two steps until the logarithmic natural function value converges; In the first step, the probability P(z k |d m ,w v ), then assume that P(w v |z k ) and P(z k |d m ) Two factors are known; In step 2, the lower limit L of the likelihood function is approximated by maximizing the expected value of the complete data likelihood: P(z k |d m ,w v ) can be obtained by calculating in step 1, P(w v |z k ) and P(z k |d m ) two factors satisfy two constraints, so the Lagrange multiplier method is used to transform the constrained optimization problem into an unconstrained optimization problem, and the objective function is v |z k ) and P(z k |d m ) and set the equation to 0 to obtain the iterative formula for the second step re-estimation: The above formula is calculated repeatedly and the initial value is continuously corrected until the iteration converges. The maximum likelihood value of each iteration gradually increases to a certain convergence value.

Images