An automatic ground object classification system for multispectral and visible light images

Abstract

The present invention discloses a multispectral and visible light image automated feature classification system, which relates to the field of remote sensing and image processing technology and includes a feature data acquisition module, a feature feature extraction module, a feature classification module, a category arbitration module, and a category mapping module. After the feature data is collected and preprocessed, the feature extraction module extracts multispectral and visible light features, the feature classification module constructs a model to output the probability distribution of various types of features, the category arbitration module intelligently decides and outputs the final category, and the category mapping module generates feature classification thematic maps and reports, and sets alarm thresholds and feedback instructions. The present invention achieves efficient fusion of multispectral and visible light images and automated feature classification, improving classification accuracy and efficiency. Through intelligent decision-making arbitration, classification errors are reduced. The generated thematic maps and reports are intuitive and easy to use. The setting of alarm thresholds and feedback instructions enhances the monitoring and response capabilities of feature changes, and has broad application prospects and value.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing and image processing, and in particular to an automatic ground object classification system for multispectral and visible light images. Background Art

[0002] Feature classification is the process of scientifically classifying relatively fixed objects on the earth's surface. These objects are divided into different categories according to their natural properties and human characteristics. Generally speaking, features can be divided into two categories: natural features and artificial features. Natural features mainly include objects in nature, such as vegetation, soil and water bodies. These features are part of the earth's natural environment and play an important role in the balance of the earth's ecosystem. Artificial features are objects formed by human activities, such as buildings, roads and bare land. These features reflect the development and progress of human society and are also an important part of human life. By classifying features, we can better understand the characteristics and properties of the earth's surface and provide a scientific basis for geographical research, urban planning, environmental protection and other fields.

[0003] In order to solve the problems of semantic conflicts and logical contradictions in complex scenes of feature classification, the existing technology mainly adopts the traditional feature classification method based on single spectrum or visible light image. However, when faced with complex and changeable feature types and complex scenes, this method often results in inaccurate classification results, misclassification, and semantic conflicts and logical contradictions between classification results. These problems in turn seriously affect the reliability and practicality of feature classification, and cannot meet the needs of high-precision feature classification and geographic information extraction. In order to overcome these limitations, an automatic feature classification system with multispectral and visible light images is proposed. Summary of the Invention

[0004] The present invention aims to provide a multi-spectral and visible light image automatic object classification system to solve the problems raised in the above background technology.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: a multispectral and visible light image automatic object classification system, including an object data acquisition module, an object feature extraction module, an object classification module, a category arbitration module and a category mapping module;

[0006] The ground object data acquisition module collects and preprocesses ground object multispectral data and visible light RGB image data;

[0007] The ground feature extraction module extracts multispectral features and visible light features based on the preprocessed ground feature multispectral data and visible light RGB image data;

[0008] The object classification module integrates multispectral features and visible light features to build spectral classification models, spatial classification models, and object classification models, and outputs the probability distribution of vegetation, soil, water, buildings, roads, and bare land, as well as the fusion probability;

[0009] The category arbitration module performs intelligent decision arbitration based on the probability distribution of vegetation, soil, water, buildings, roads and bare land, as well as the fusion probability, and outputs the final category;

[0010] The category mapping module maps the final category to the geographic information system, generates a feature classification thematic map, presets rules to derive classification statistical reports, sets feature category area change rate alarm thresholds, and generates feedback instructions.

[0011] A further improvement of the technical solution of the present invention is that: in the ground object data acquisition module, the acquisition and preprocessing process of ground object multispectral data and visible light RGB image data includes:

[0012] The drone platform is equipped with a multispectral imager and a visible light camera. The multispectral imager is equipped with near-infrared, red, green, and blue light band sensors, with a spectral resolution of less than 10 nanometers and a spatial resolution of 10 meters. The visible light camera uses a three-channel charge-coupled device with a spatial resolution of 0.5 meters and a frame rate of no less than 30 frames per second. It also has a built-in time synchronizer and global positioning module.

[0013] The multispectral imager divides the incident light into near-infrared, red, green, and blue bands. Detectors in each band simultaneously record the reflected energy of ground objects, generating a radiation brightness matrix to collect multispectral data of ground objects. The drone's trajectory is then planned to ensure that the overlap rate of adjacent images exceeds 30%. The visible light camera uses a Bayer filter to separate the light, recording the light intensity of the red, green, and blue bands respectively, and outputs a three-channel digital matrix to collect visible light RGB image data.

[0014] Radiometric correction and geometric correction are performed on the collected multispectral data and visible light RGB image data. The dark current subtraction method is used to eliminate the influence of atmospheric scattering and compensate for the difference in solar altitude angle. An affine transformation model is established based on ground control points to achieve coordinate alignment. The multispectral data is converted into a hierarchical data format, and the visible light image is converted into a geo-tagged image format, which are then transmitted to the ground feature extraction module via the message queue.

[0015] A further improvement of the technical solution of the present invention is that: in the ground feature extraction module, the process of extracting multispectral features and visible light features includes:

[0016] Multispectral features include vegetation index features and spectral reflectance features, and visible light features include spatial texture features and color distribution features;

[0017] Based on the reflectance of the near-infrared and red bands of multispectral data, the normalized vegetation index is obtained through its normalized difference, the vegetation growth status is quantified, and the vegetation index characteristics are extracted;

[0018] The mean reflectance of the green light band is calculated to represent the vegetation activity. The standard deviation of the reflectance in the near-infrared band is used to describe the spectral fluctuation characteristics, and a three-dimensional spectral feature vector is constructed to extract the spectral reflectance characteristics.

[0019] Convert the visible light RGB image into grayscale image, calculate the contrast and entropy using the grayscale co-occurrence matrix of the local window, and extract the spatial texture features;

[0020] Convert the preprocessed visible light RGB image to the HSV color space, extract the hue peak to determine the dominant color, describe the discrete degree of color distribution according to the saturation standard deviation, form a two-dimensional color feature vector, and extract the color distribution characteristics;

[0021] Multispectral features and visible light features are normalized to their maximum and minimum values, respectively. Multispectral features are stored as floating-point arrays, and visible light features are encapsulated as structured data packets. Bilinear interpolation is used to unify multispectral features and visible light features to a 0.5-meter grid. When the normalized vegetation index is greater than 1 and the hue peak is less than 0, abnormal pixels are automatically removed and the collected data is re-extracted.

[0022] A further improvement of the technical solution of the present invention is that: in the object classification module, the process of constructing a spectral classification model and outputting the probability distribution of vegetation, soil and water includes:

[0023] The normalized difference vegetation index, the mean of the green band, and the standard deviation of the near-infrared band are integrated into a three-dimensional node feature vector, and the prior probabilities of vegetation, soil, and water in the training samples are used as the initial prediction values.

[0024] Based on the gradient boosting decision tree algorithm, each decision tree is trained with the prediction residual of the previous decision tree as the training target. The node feature that reduces the Gini impurity the most is selected for splitting. The maximum depth of each decision tree is set to 6 layers, and the minimum number of leaf node samples is set to 10. The spectral classification model is constructed.

[0025] The terminal leaf node of each decision tree stores the distribution weights of vegetation, soil, and water bodies of the samples in the node. The final output is the cumulative value of the vegetation, soil, and water body weights in 500 decision trees. The cumulative value of the vegetation, soil, and water body weights of the 500 decision trees is converted into the probability distribution of vegetation, soil, and water bodies through the Softmax function, and the sum of the vegetation, soil, and water body probabilities is 1.

[0026] A further improvement of the technical solution of the present invention is that: in the object classification module, the process of constructing a spatial classification model and outputting the probability distribution of buildings, roads and bare land includes:

[0027] The contrast and entropy generated in the process of extracting spatial texture features and the hue peak and saturation standard deviation generated in the process of extracting color distribution features are combined into a four-dimensional input feature vector;

[0028] Based on a depthwise separable convolutional architecture, a spatial classification model is constructed. The spatial classification model performs independent spatial convolution on each input feature channel, extracts spatial information from each input feature channel, fuses features across channels through 1×1 point convolution, forms a 32-dimensional feature map, uses global average pooling to map the 32-dimensional features into a global feature vector, and compresses the global feature vector to 3 dimensions through a fully connected layer, with each dimension corresponding to a building, road, and bare land score.

[0029] Softmax normalization is performed on the building, road, and bare land scores output by the spatial classification model to convert the linear scores into probability distributions of buildings, roads, and bare land, and the sum of the probabilities of buildings, roads, and bare land is 1.

[0030] The weighted cross entropy loss function is used to dynamically adjust the weight according to the number of category samples, giving higher weights to categories with small samples. The initial value of the learning rate is set to 0.001, and the learning rate is adjusted in combination with the adaptive moment estimation optimizer. The L2 regularization is added to constrain the convolution kernel parameters.

[0031] A further improvement of the technical solution of the present invention is that: in the object classification module, the process of constructing the object classification model and outputting the fusion probability includes:

[0032] The probability distribution of vegetation, soil and water output by the spectral classification model , the probability distribution of buildings, roads and bare land output by the spatial classification model Merge into a six-dimensional joint feature vector F, introduce a gated fusion unit, and dynamically calculate the contribution weights of spectral and spatial features , build a ground feature classification model and output the fusion probability , the calculation process is as follows:

[0033] ;

[0034] ;

[0035] in, is the Sigmoid function, and is a trainable parameter, .

[0036] A further improvement of the technical solution of the present invention is that: in the category arbitration module, the process of performing intelligent decision arbitration and outputting the final category includes:

[0037] Semantic compatibility rules are set: vegetation and water bodies cannot coexist with buildings, and roads and bare land cannot coexist with water bodies. When the probability of any category in the fusion probability reaches 0.9, the result is directly output. For areas where the fusion probability does not reach 0.9, if the output results of the spectral classification model and the spatial classification model meet the semantic compatibility rules, the category with the highest corresponding probability is selected. If the output results of the spectral classification model and the spatial classification model conflict with the semantic compatibility rules, they are marked as unknown, and the vegetation, soil, and water categories are prioritized.

[0038] The maximum probability product of the spectrum and the spatial model is calculated by geometric averaging as the comprehensive confidence C. The calculation process is as follows:

[0039] ;

[0040] If C reaches 0.85, it is marked as high confidence and the highest probability category is output. If C is between 0.6 and 0.85, the spectral category is dominant. If C is less than 0.6, it is an unknown category pixel, and a secondary judgment based on near-infrared reflectivity and texture entropy is prioritized. If the near-infrared reflectivity is greater than 0.4, it is classified as vegetation. If the texture entropy value is greater than 1.5, it is classified as road. If it still cannot be classified, the corresponding coordinates are recorded and the drone is triggered to re-sample the area.

[0041] According to the accuracy difference of the validation set between the spectral and spatial models, the fusion weight is dynamically adjusted with a learning rate of 0.01, so that the high-precision land object classification model has a higher decision weight in arbitration.

[0042] A further improvement of the technical solution of the present invention is that: in the category mapping module, the process of mapping the final category to the geographic information system to generate the feature classification thematic map includes:

[0043] receiving a final classification result and corresponding pixel coordinates, and introducing affine transformation parameters to map the pixel coordinates to a geographic coordinate system, wherein the final classification result includes vegetation, soil, water, building, road, bare land, and unknown;

[0044] Generate a feature classification thematic map with associated category name, confidence level, acquisition timestamp, and conflict type labeled. Fill the feature classification thematic map with different colors corresponding to different categories. Map the confidence level to transparency. Overlay the classification results with the digital elevation model to generate a three-dimensional feature distribution map. The three-dimensional feature distribution map is colored by elevation layer.

[0045] When the confidence level of the new data is higher than the old value, the original result is overwritten and a time series change layer is generated. The area change rate of the feature in the adjacent periods is calculated. The area where the area change rate exceeds 5% is marked with a flashing effect.

[0046] The feature classification thematic map is stored in a block pyramid structure, and the attribute data is indexed by the geographic coordinate hash value. It supports the web map service protocol, returns the raster slices of the specified area and level on demand, and provides a coordinate query interface to return the feature category and confidence level.

[0047] Generate an independent unknown layer for manual review, and automatically associate the new and old coordinates after triggering local re-sampling.

[0048] A further improvement of the technical solution of the present invention is that: in the category mapping module, the process of presetting rules to derive classification statistical reports includes:

[0049] The number of pixels is counted by feature category, the actual area is calculated based on the spatial resolution, and the area change rates of adjacent periods are compared. A classification statistical report containing the category area, area change rates of adjacent periods, and confidence levels is automatically generated every day.

[0050] A further improvement of the technical solution of the present invention is that: in the category mapping module, the process of setting the alarm threshold of the area change rate of the feature category and generating the feedback instruction includes:

[0051] Set area thresholds for vegetation, soil, water, buildings, roads, bare land, and unknown areas. When the area change rate in adjacent cycles exceeds the corresponding area threshold for three consecutive cycles, an alarm is triggered and the exceeded area is marked.

[0052] If the unknown area accounts for more than 5% and the confidence mean is less than 0.7, a local re-sampling instruction is generated, and the target area is the abnormal coordinate set. If the vegetation area decreases by more than 10% and the water area decreases by more than 5%, it is judged as vegetation degradation and the irrigation instruction is triggered. If the building and road area increases by more than 5% and the bare land area increases by more than 15%, it is judged as building encroachment, triggering an enforcement inspection instruction and marking the coordinates of the encroached area.

[0053] Due to the adoption of the above technical solution, the present invention has the following technical advancements compared to the prior art:

[0054] 1. The present invention provides an automated land feature classification system based on multispectral and visible light images. By integrating multispectral and visible light data, the accuracy and efficiency of land feature classification are significantly improved, providing strong support for the rapid acquisition and analysis of geographic spatial information.

[0055] 2. The present invention provides an automatic land object classification system based on multispectral and visible light images. By using an intelligent decision-making arbitration module, it effectively solves the land object classification problem in complex scenarios, reduces classification errors, and improves the reliability and practicality of classification results.

[0056] 3. The present invention provides an automatic land feature classification system based on multispectral and visible light images. Through the category mapping module, it can automatically generate land feature classification thematic maps and classification statistical reports, and at the same time set the alarm threshold for the area change rate of land feature categories, providing convenient and efficient data support for geographical environment monitoring and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0058] Figure 1 A block diagram of the present invention. DETAILED DESCRIPTION

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0060] Example 1, as Figure 1 As shown, the present invention provides a multi-spectral and visible light image automatic object classification system, including an object data acquisition module, an object feature extraction module, an object classification module, a category arbitration module and a category mapping module;

[0061] The ground object data acquisition module collects and pre-processes ground object multispectral data and visible light RGB image data. The UAV platform is equipped with a multispectral imager and a visible light camera. The multispectral imager is equipped with near-infrared, red, green and blue light band sensors with a spectral resolution of less than 10 nanometers and a spatial resolution of 10 meters. The visible light camera uses a three-channel charge-coupled device with a spatial resolution of 0.5 meters and a frame rate of not less than 30 frames per second. It also has a built-in time synchronizer and global positioning module. The multispectral imager divides the incident light into near-infrared, red, green and blue light bands. The detectors in each band synchronously record the reflected energy of the ground object to generate a radiation brightness matrix to collect multi-spectral data of the ground object. spectral data, and through UAV trajectory planning, the overlap rate of adjacent images is higher than 30%. The visible light camera uses Bayer filter to split the light, record the light intensity of the red, green and blue light bands respectively, output a three-channel digital matrix, collect visible light RGB image data, perform radiation correction and geometric correction on the collected multispectral data of the ground object and the visible light RGB image data, use the dark current subtraction method to eliminate the influence of atmospheric scattering and compensate for the difference in solar altitude angle, establish an affine transformation model based on the ground control points to achieve coordinate alignment, convert the multispectral data into a hierarchical data format, and convert the visible light image into a geo-tagged image format, which are then transmitted to the ground object feature extraction module via the message queue;

[0062] The ground feature extraction module extracts multispectral features and visible light features based on the pre-processed ground feature multispectral data and visible light RGB image data. Multispectral features include vegetation index features and spectral reflectance features, and visible light features include spatial texture features and color distribution features. Based on the near-infrared and red light band reflectance of multispectral data, the normalized vegetation index is obtained through its normalized difference, the vegetation growth state is quantified, the vegetation index features are extracted, the mean green light band reflectance is calculated to characterize the vegetation activity, the spectral fluctuation characteristics are described according to the standard deviation of the near-infrared band reflectance, a three-dimensional spectral feature vector is constructed, the spectral reflectance features are extracted, and the visible light RGB image is converted into a grayscale image. The contrast and entropy of the gray-level co-occurrence matrix of the local window are calculated to extract spatial texture features. The preprocessed visible light RGB image is converted to the HSV color space. The hue peak is extracted to determine the dominant color. The discrete degree of color distribution is described according to the saturation standard deviation to form a two-dimensional color feature vector. The color distribution characteristics are extracted. The multispectral features and visible light features are normalized to the maximum and minimum values ​​respectively. The multispectral features are stored as floating-point arrays, and the visible light features are encapsulated as structured data packets. The multispectral features and visible light features are unified to a 0.5-meter grid using bilinear interpolation. When the normalized vegetation index is greater than 1 and the hue peak is less than 0, abnormal pixels are automatically removed and the collected data is re-extracted.

[0063] The object classification module integrates multispectral features and visible light features to build spectral classification models, spatial classification models and object classification models, outputs the probability distribution of vegetation, soil, water, buildings, roads and bare land, as well as the fusion probability, integrates the normalized difference vegetation index, the mean of the green light band and the standard deviation of the near-infrared band into a three-dimensional node feature vector, uses the prior probability of vegetation, soil and water in the training sample as the initial prediction value, and uses the gradient boosting decision tree algorithm. Each decision tree uses the prediction residual of the previous decision tree as the training target, and selects the one that reduces the Gini impurity the most. The node features are split, the maximum depth of each decision tree is set to 6 layers, the minimum number of leaf node samples is 10, and the spectral classification model is constructed. The terminal leaf node of each decision tree saves the distribution weights of vegetation, soil and water bodies of the samples in the node. The final output is the cumulative value of the vegetation, soil and water body weights in 500 decision trees. The cumulative value of the vegetation, soil and water body weights of 500 decision trees is converted into the probability distribution of vegetation, soil and water bodies through the Softmax function, and the sum of the probability of vegetation, soil and water bodies is 1. The generated contrast, entropy, and hue peak and saturation standard deviation generated in the process of extracting color distribution features are combined into a four-dimensional input feature vector. Based on the depthwise separable convolution architecture, a spatial classification model is constructed. The spatial classification model performs spatial convolution on each input feature channel independently to extract the spatial information of each input feature channel. The features are fused across channels through 1×1 point convolution to form a 32-dimensional feature map. The 32-dimensional feature is mapped to a global feature vector using global average pooling. The global feature vector is compressed to 3 dimensions through a fully connected layer, and each dimension corresponds to a building, road, and bare land score. The building, road, and bare land scores output by the spatial classification model are softmax normalized, and the linear scores are converted into probability distributions of buildings, roads, and bare land, and the sum of the probabilities of buildings, roads, and bare land is 1. The weighted cross entropy loss function is used to dynamically adjust the weights according to the number of category samples, giving higher weights to small sample categories. The initial learning rate is set to 0.001, and the learning rate is adjusted in combination with the adaptive moment estimation optimizer. The convolution kernel parameters are constrained with L2 regularization. The probability distributions of vegetation, soil, and water output by the spectral classification model are converted to , the probability distribution of buildings, roads and bare land output by the spatial classification model Merge into a six-dimensional joint feature vector F, introduce a gated fusion unit, and dynamically calculate the contribution weights of spectral and spatial features , build a ground feature classification model and output the fusion probability , the calculation process is as follows:

[0064] ;

[0065] ;

[0066] in, is the Sigmoid function, and is a trainable parameter, ;

[0067] The category arbitration module performs intelligent decision arbitration based on the probability distribution of vegetation, soil, water, buildings, roads, and bare land, as well as the fusion probability, and outputs the final category. It sets semantic compatibility rules: vegetation and water cannot coexist with buildings, and roads and bare land cannot coexist with water. When the probability of any category in the fusion probability reaches 0.9, the result is directly output. For areas where the fusion probability does not reach 0.9, if the output results of the spectral classification model and the spatial classification model meet the semantic compatibility rules, the category with the highest corresponding probability is selected. If the output results of the spectral classification model and the spatial classification model conflict with the semantic compatibility rules, they are marked as unknown, and the vegetation, soil, and water categories are determined first. The maximum probability product of the spectral and spatial models is calculated by geometric averaging as the comprehensive confidence C. The calculation process is as follows:

[0068] ;

[0069] If C reaches 0.85, it is marked as high confidence and the highest probability category is output. If C is between 0.6 and 0.85, the spectral category is dominant. If C is less than 0.6, it is an unknown category pixel, and a secondary judgment based on near-infrared reflectivity and texture entropy is prioritized. If the near-infrared reflectivity is greater than 0.4, it is classified as vegetation. If the texture entropy is greater than 1.5, it is classified as road. If classification is still unreliable, the corresponding coordinates are recorded and the drone is triggered to resample the area. Based on the difference in validation set accuracy between the spectral and spatial models, the fusion weight is dynamically adjusted with a learning rate of 0.01, so that the high-precision object classification model has a higher decision weight in arbitration.

[0070] The category mapping module maps the final category to the geographic information system, generates a feature classification thematic map, presets rules to export classification statistical reports, sets the feature category area change rate alarm threshold, and generates feedback instructions, receives the final classification results and corresponding pixel coordinates, introduces affine transformation parameters to map the pixel coordinates to the geographic coordinate system. The final classification results include vegetation, soil, water bodies, buildings, roads, bare land and unknown, generates a feature classification thematic map with associated category names, confidence levels, acquisition timestamps and conflict types, fills the feature classification thematic map with different colors corresponding to different categories, maps confidence levels to transparency, overlays the classification results with the digital elevation model, generates a three-dimensional feature distribution map, and colors the three-dimensional feature distribution map according to elevation layers. When the confidence level of the new data is higher than the old value, the original result is overwritten. A time series change layer is generated, and the feature area change rate of adjacent periods is calculated. Areas where the feature area change rate exceeds 5% are marked with a flashing effect. The feature classification thematic map is stored in a block pyramid structure, and attribute data is indexed by geographic coordinate hash values, supporting network map services. The system automatically generates a daily statistical report containing the category area, area change rate of adjacent periods, and confidence level. Area thresholds are set for vegetation, soil, water, buildings, roads, bare land, and unknown areas. When the area change rate of adjacent periods exceeds the corresponding area threshold for three consecutive periods, an alarm is triggered and the exceeded area is marked. If the proportion of unknown areas exceeds 5% and the average confidence level is less than 0.7, a local re-sampling instruction is generated, with the target area being the set of abnormal coordinates. If the vegetation area decreases by more than 10% and the water area decreases by more than 5%, it is judged as vegetation degradation, triggering an irrigation instruction. If the building and road area increases by more than 5% and the bare land area increases by more than 15%, it is judged as building encroachment, triggering an enforcement inspection instruction, and marking the coordinates of the encroached area.

[0071] First, the multispectral data and visible light RGB image data of the required area are collected through the feature data acquisition module, and preprocessed to improve the data quality. Then, the multispectral features and visible light features are extracted from the preprocessed data using the feature feature extraction module. Then, the feature classification module integrates these features to construct a classification model and outputs the probability distribution and fusion probability of vegetation, soil, water bodies, buildings, roads and bare land. Then, the category arbitration module performs intelligent decision arbitration based on these probability distributions to determine the final feature category. Finally, the category mapping module maps the final category into the geographic information system, generates a feature classification thematic map, and exports classification statistical reports according to preset rules. In addition, the alarm threshold of the feature category change rate can also be set. When the category change is detected to exceed the threshold, the system will automatically generate feedback instructions so that corresponding measures can be taken in time. The entire process realizes integrated automated processing from data collection to application of classification results.

[0072] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A multispectral and visible light image automatic feature classification system, characterized by: It includes a ground object data acquisition module, a ground object feature extraction module, a ground object classification module, a category arbitration module and a category mapping module; The ground object data acquisition module collects and preprocesses ground object multispectral data and visible light RGB image data; The ground feature extraction module extracts multispectral features and visible light features based on the preprocessed ground feature multispectral data and visible light RGB image data; The object classification module integrates multispectral features and visible light features to build spectral classification models, spatial classification models, and object classification models, and outputs the probability distribution of vegetation, soil, water, buildings, roads, and bare land, as well as the fusion probability; The category arbitration module performs intelligent decision arbitration based on the probability distribution of vegetation, soil, water, buildings, roads and bare land, as well as the fusion probability, and outputs the final category; The category mapping module maps the final category to the geographic information system, generates a feature classification thematic map, presets rules to derive classification statistical reports, sets feature category area change rate alarm thresholds, and generates feedback instructions; In the object classification module, the process of constructing an object classification model and outputting a fusion probability includes: The probability distribution of vegetation, soil and water output by the spectral classification model , the probability distribution of buildings, roads and bare land output by the spatial classification model Merge into a six-dimensional joint feature vector F, introduce a gated fusion unit, and dynamically calculate the contribution weights of spectral and spatial features , build a ground feature classification model and output the fusion probability ; In the category arbitration module, the process of performing intelligent decision arbitration and outputting the final category includes: Semantic compatibility rules are set: vegetation and water bodies cannot coexist with buildings, and roads and bare land cannot coexist with water bodies. When the probability of any category in the fusion probability reaches 0.9, the result is directly output. For areas where the fusion probability does not reach 0.9, if the output results of the spectral classification model and the spatial classification model meet the semantic compatibility rules, the category with the highest corresponding probability is selected. If the output results of the spectral classification model and the spatial classification model conflict with the semantic compatibility rules, they are marked as unknown, and the vegetation, soil, and water categories are prioritized. The maximum probability product of the spectral classification model and the spatial classification model is calculated by geometric averaging as the comprehensive confidence C. The calculation process is as follows: ； If C reaches 0.85, it is marked as high confidence and the highest probability category is output. If C is between 0.6 and 0.85, the spectral category is dominant. If C is less than 0.6, it is an unknown category pixel, and a secondary judgment based on near-infrared reflectivity and texture entropy is prioritized. If the near-infrared reflectivity is greater than 0.4, it is classified as vegetation. If the texture entropy value is greater than 1.5, it is classified as road. If it still cannot be classified, the corresponding coordinates are recorded and the drone is triggered to re-sample the area. According to the accuracy difference of the validation set between the spectral and spatial models, the fusion weight is dynamically adjusted with a learning rate of 0.01, so that the high-precision land object classification model has a higher decision weight in arbitration.

2. The multispectral and visible light image automatic object classification system according to claim 1, characterized in that: In the object data acquisition module, the acquisition and preprocessing process of the object multispectral data and visible light RGB image data includes: The UAV platform is equipped with a multispectral imager and a visible light camera. The multispectral imager divides the incident light into near-infrared, red, green, and blue bands. Detectors in each band synchronously record the reflected energy of the ground objects, generate a radiation brightness matrix, and collect multispectral data of the ground objects. The visible light camera uses a Bayer filter to split the light, record the light intensity of the red, green, and blue bands respectively, and output a three-channel digital matrix to collect visible light RGB image data. Radiometric correction and geometric correction are performed on the collected multispectral data and visible light RGB image data to eliminate the influence of atmospheric scattering and compensate for the difference in solar altitude angle. An affine transformation model is established based on ground control points to achieve coordinate alignment. The multispectral data is converted into a hierarchical data format, and the visible light image is converted into a geo-tagged image format, which are then transmitted to the ground feature extraction module via the message queue.

3. The multispectral and visible light image automatic object classification system according to claim 2, characterized in that: In the ground feature extraction module, the process of extracting multispectral features and visible light features includes: Multispectral features include vegetation index features and spectral reflectance features, and visible light features include spatial texture features and color distribution features; Based on the reflectance of the near-infrared and red bands of multispectral data, the normalized vegetation index is obtained through its normalized difference, the vegetation growth status is quantified, and the vegetation index characteristics are extracted; The mean reflectance of the green light band is calculated to represent the vegetation activity. The standard deviation of the reflectance in the near-infrared band is used to describe the spectral fluctuation characteristics, and a three-dimensional spectral feature vector is constructed to extract the spectral reflectance characteristics. Convert the visible light RGB image into grayscale image, calculate the contrast and entropy using the grayscale co-occurrence matrix of the local window, and extract the spatial texture features; Convert the preprocessed visible light RGB image to the HSV color space, extract the hue peak to determine the dominant color, describe the discrete degree of color distribution according to the saturation standard deviation, form a two-dimensional color feature vector, and extract the color distribution characteristics; Multispectral features and visible light features are normalized to their maximum and minimum values, respectively. Multispectral features are stored as floating-point arrays, and visible light features are encapsulated as structured data packets. Bilinear interpolation is used to unify multispectral features and visible light features to a 0.5-meter grid. When the normalized vegetation index is greater than 1 and the hue peak is less than 0, abnormal pixels are automatically removed and the collected data is re-extracted.

4. The multispectral and visible light image automatic object classification system according to claim 3, characterized in that: In the object classification module, the process of constructing a spectral classification model and outputting the probability distribution of vegetation, soil, and water includes: The normalized difference vegetation index, the mean of the green band, and the standard deviation of the near-infrared band are integrated into a three-dimensional node feature vector, and the prior probabilities of vegetation, soil, and water in the training samples are used as the initial prediction values. Based on the gradient boosting decision tree algorithm, each decision tree is trained with the prediction residual of the previous decision tree as the training target. The node feature that reduces the Gini impurity the most is selected for splitting. The maximum depth of each decision tree is set to 6 layers, and the minimum number of leaf node samples is set to 10. The spectral classification model is constructed. The terminal leaf node of each decision tree stores the distribution weights of vegetation, soil, and water bodies of the samples in the node. The final output is the cumulative value of the vegetation, soil, and water body weights in 500 decision trees. The cumulative value of the vegetation, soil, and water body weights of the 500 decision trees is converted into the probability distribution of vegetation, soil, and water bodies through the Softmax function, and the sum of the vegetation, soil, and water body probabilities is 1.

5. The multispectral and visible light image automatic object classification system according to claim 4, characterized in that: In the object classification module, the process of constructing a spatial classification model and outputting the probability distribution of buildings, roads, and bare land includes: The contrast and entropy generated in the process of extracting spatial texture features and the hue peak and saturation standard deviation generated in the process of extracting color distribution features are combined into a four-dimensional input feature vector; Based on a depthwise separable convolutional architecture, a spatial classification model is constructed. The spatial classification model performs independent spatial convolution on each input feature channel, extracts spatial information from each input feature channel, fuses features across channels through 1×1 point convolution, forms a 32-dimensional feature map, uses global average pooling to map the 32-dimensional features into a global feature vector, and compresses the global feature vector to 3 dimensions through a fully connected layer, with each dimension corresponding to a building, road, and bare land score. Softmax normalization is performed on the building, road, and bare land scores output by the spatial classification model to convert the linear scores into probability distributions of buildings, roads, and bare land, and the sum of the probabilities of buildings, roads, and bare land is 1. The weighted cross entropy loss function is used to dynamically adjust the weight according to the number of category samples, giving higher weights to categories with small samples. The initial value of the learning rate is set to 0.001, and the learning rate is adjusted in combination with the adaptive moment estimation optimizer. The L2 regularization is added to constrain the convolution kernel parameters.

6. The multispectral and visible light image automatic object classification system according to claim 5, characterized in that: In the category mapping module, the process of mapping the final category to the geographic information system and generating the feature classification thematic map includes: receiving a final classification result and corresponding pixel coordinates, and introducing affine transformation parameters to map the pixel coordinates to a geographic coordinate system, wherein the final classification result includes vegetation, soil, water, building, road, bare land, and unknown; Generate a feature classification thematic map with associated category name, confidence level, acquisition timestamp, and conflict type labeled. Fill the feature classification thematic map with different colors corresponding to different categories. Map the confidence level to transparency. Overlay the classification results with the digital elevation model to generate a three-dimensional feature distribution map. The three-dimensional feature distribution map is colored by elevation layer. When the confidence level of the new data is higher than the old value, the original result is overwritten and a time series change layer is generated. The area change rate of the feature in the adjacent periods is calculated. The area where the area change rate exceeds 5% is marked with a flashing effect. The feature classification thematic map is stored in a block pyramid structure, and the attribute data is indexed by the geographic coordinate hash value. It supports the web map service protocol, returns the raster slices of the specified area and level on demand, and provides a coordinate query interface to return the feature category and confidence level. Generate an independent unknown layer for manual review, and automatically associate the new and old coordinates after triggering local re-sampling.

7. The multispectral and visible light image automatic object classification system according to claim 6, characterized in that: In the category mapping module, the process of presetting rules to export classification statistical reports includes: The number of pixels is counted by feature category, the actual area is calculated based on the spatial resolution, and the area change rates of adjacent periods are compared. A classification statistical report containing the category area, area change rates of adjacent periods, and confidence levels is automatically generated every day.

8. The multispectral and visible light image automatic object classification system according to claim 7, characterized in that: In the category mapping module, the process of setting the alarm threshold of the area change rate of the feature category and generating the feedback instruction includes: Set area thresholds for vegetation, soil, water, buildings, roads, bare land, and unknown areas. When the area change rate in adjacent cycles exceeds the corresponding area threshold for three consecutive cycles, an alarm is triggered and the exceeded area is marked. If the unknown area accounts for more than 5% and the confidence mean is less than 0.7, a local re-sampling instruction is generated, and the target area is the abnormal coordinate set. If the vegetation area decreases by more than 10% and the water area decreases by more than 5%, it is judged as vegetation degradation and the irrigation instruction is triggered. If the building and road area increases by more than 5% and the bare land area increases by more than 15%, it is judged as building encroachment, triggering an enforcement inspection instruction and marking the coordinates of the encroached area.

Images