Multispectral and visible light image automatic ground object classification system

Abstract

The invention, which relates to the technical field of remote sensing and image processing, discloses a multispectral and visible light image automatic surface feature classification system comprising a surface feature data acquisition module, a surface feature feature extraction module, a surface feature classification module, a category arbitration module and a category mapping module. After surface feature data are collected and preprocessed, a feature extraction module extracts multispectral and visible light features, a surface feature classification module constructs a model to output probability distribution of various surface features, a category arbitration module intelligently decides and outputs a final category, and a category mapping module generates a surface feature classification thematic map and a report and sets an alarm threshold and a feedback instruction. According to the method, efficient fusion of multispectral and visible light images and automatic ground object classification are realized, the classification precision and efficiency are improved, classification errors are reduced through intelligent decision arbitration, generated thematic maps and reports are visual and easy to use, the monitoring and response capability of ground object changes is enhanced through setting of an alarm threshold value and a feedback instruction, and the method is suitable for large-scale popularization and application. Wide application prospects and values are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing and image processing, and particularly to a multi-spectral and visible light image automatic ground object classification system. Background Art

[0002] Ground object classification is the process of scientifically classifying relatively fixed objects on the Earth's surface. These objects are divided into different categories according to their natural attributes and artificial characteristics. Generally speaking, ground objects can be divided into two major categories: natural ground objects and artificial ground objects. Natural ground objects mainly include objects in nature, such as vegetation, soil, and water bodies. These ground objects are components of the Earth's natural environment and play an important role in maintaining the balance of the Earth's ecological system. Artificial ground objects are objects formed by human activities, such as buildings, roads, and bare land. These ground objects reflect the development and progress of human society and are also important components of human life. By classifying ground objects, we can better understand the characteristics and attributes of the Earth's surface and provide a scientific basis for fields such as geographical research, urban planning, and environmental protection.

[0003] In order to solve the problems of semantic conflicts and logical contradictions of ground objects in complex scenarios of ground object classification, the existing technology mainly adopts the traditional ground object classification method based on single-spectrum or visible light images. However, when facing complex and variable ground object types and complex scenarios, this method often results in inaccurate classification results, misclassification, and semantic conflicts and logical contradictions between classification results. These problems further seriously affect the reliability and practicality of ground object classification and cannot meet the requirements of high-precision ground object classification and geographical information extraction. To overcome these limitations, a multi-spectral and visible light image automatic ground object classification system is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-spectral and visible light image automatic ground object classification system to solve the problems mentioned in the above background art.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is: a multi-spectral and visible light image automatic ground object classification system, including a ground object data acquisition module, a ground object feature extraction module, a ground object classification module, a category arbitration module, and a category mapping module;

[0006] The ground object data acquisition module collects and preprocesses ground object multi-spectral data and visible light RGB image data;

[0007] The ground object feature extraction module extracts multi-spectral features and visible light features based on the preprocessed ground object multi-spectral data and visible light RGB image data;

[0008] The ground object classification module comprehensively combines multi-spectral features and visible light features to construct a spectral classification model, a spatial classification model, and a ground object classification model, and outputs the probability distributions of vegetation, soil, water bodies, buildings, roads, and bare land, as well as the fusion probability;

[0009] The category arbitration module performs intelligent decision arbitration based on the probability distributions of vegetation, soil, water bodies, buildings, roads, and bare land, as well as the fusion probability, and outputs the final category;

[0010] The category mapping module maps the final category to a geographic information system, generates a thematic map of ground object classification, presets rules to export classification statistical reports, sets an alarm threshold for the area change rate of ground object categories, and generates a feedback instruction.

[0011] A further improvement of the technical solution of the present invention lies in: in the ground object data acquisition module, the acquisition and preprocessing process of ground object multi-spectral data and visible light RGB image data includes:

[0012] A multi-spectral imager and a visible light camera are carried on a drone platform. Among them, the multi-spectral imager is equipped with near-infrared, red, green, and blue band sensors, with a spectral resolution of less than 10 nanometers, a spatial resolution of 10 meters, the visible light camera uses a three-channel charge-coupled device, with a spatial resolution of 0.5 meters, a frame rate of not less than 30 frames per second, and a built-in time synchronizer and a global positioning module;

[0013] The multi-spectral imager divides the incident light according to the near-infrared, red, green, and blue bands, and the detectors of each band synchronously record the reflected energy of the ground object to generate a radiance matrix, collect the ground object multi-spectral data, and through the drone flight path planning, make the overlapping rate of adjacent images higher than 30%. The visible light camera uses a Bayer filter to split the light, and respectively records the light intensities of the red, green, and blue bands, and outputs a three-channel digital matrix to collect the visible light RGB image data;

[0014] Perform radiometric correction and geometric correction on the collected ground object multi-spectral data and visible light RGB image data, use the dark current subtraction method to eliminate the influence of atmospheric scattering and compensate for the difference in solar altitude angle, establish an affine transformation model based on ground control points to achieve coordinate registration, convert the multi-spectral data into a hierarchical data format, and convert the visible light image into a geotagged image format, and transmit it to the ground object feature extraction module through a message queue.

[0015] A further improvement of the technical solution of the present invention lies in: in the ground object feature extraction module, the process of extracting multi-spectral features and visible light features includes:

[0016] The multi-spectral features include vegetation index features and spectral reflection features, and the visible light features include spatial texture features and color distribution features;

[0017] Based on the reflectance of the near-infrared and red light bands in the multispectral data, the normalized difference vegetation index is obtained through its normalization difference to quantify the vegetation growth state and extract the vegetation index characteristics;

[0018] The mean reflectance of the green light band is statistically characterized to represent the vegetation activity, and the spectral fluctuation characteristics are described according to the standard deviation of the near-infrared band reflectance. A three-dimensional spectral feature vector is constructed to extract the spectral reflection characteristics;

[0019] The visible light RGB image is converted into a grayscale image, and the contrast and entropy are calculated using the gray-level co-occurrence matrix of the local window to extract the spatial texture characteristics;

[0020] The preprocessed visible light RGB image is converted to the HSV color space, the hue peak value is extracted to determine the main color, and the discrete degree of the color distribution is described according to the standard deviation of the saturation. A two-dimensional color feature vector is formed to extract the color distribution characteristics;

[0021] The multispectral features and visible light features are respectively normalized by the maximum-minimum normalization method. The multispectral features are stored as floating-point arrays, and the visible light features are encapsulated into structured data packets. The bilinear interpolation method is used to unify the multispectral features and visible light features to a 0.5-meter grid. When the normalized difference vegetation index is greater than 1 and the hue peak value is less than 0, abnormal pixels are automatically removed and the acquisition data is re-extracted.

[0022] A further improvement of the technical solution of the present invention lies in: in the ground object classification module, the process of constructing a spectral classification model and outputting the probability distributions of vegetation, soil, and water bodies includes:

[0023] The normalized difference vegetation index, the mean value of the green light band, and the standard deviation of the near-infrared band are integrated into a three-dimensional node feature vector, and the prior probabilities of vegetation, soil, and water bodies in the training samples are used as the initial prediction values;

[0024] Based on the decision tree algorithm of gradient boosting, each decision tree uses the prediction residual of the previous decision tree as the training target, selects the node feature that maximally reduces the Gini impurity for splitting, sets the maximum depth of each decision tree to 6 layers, and the minimum number of samples in the leaf nodes to 10, to construct a spectral classification model;

[0025] The end leaf nodes of each decision tree save the distribution weights of vegetation, soil, and water bodies of the samples in the node. The final output is the cumulative value of the weights of vegetation, soil, and water bodies in 500 decision trees. The cumulative values of the weights of vegetation, soil, and water bodies in 500 decision trees are converted into the probability distributions of vegetation, soil, and water bodies through the Softmax function, and the sum of the probabilities of vegetation, soil, and water bodies is 1.

[0026] A further improvement of the technical solution of the present invention lies in: in the ground object classification module, the process of constructing a spatial classification model and outputting the probability distributions of buildings, roads, and bare land includes:

[0027] Combine the contrast generated during the process of extracting spatial texture features, entropy, the hue peak generated during the process of extracting color distribution features, and the saturation standard deviation into a four-dimensional input feature vector;

[0028] Based on the depthwise separable convolution architecture, construct a spatial classification model. The spatial classification model independently performs spatial convolution on each input feature channel, extracts the spatial information of each input feature channel, fuses the features across channels through 1×1 point convolution to form a 32-dimensional feature map, uses global average pooling to map the 32-dimensional feature map into a global feature vector, and compresses the global feature vector to 3 dimensions through a fully connected layer. Each dimension corresponds to the scores of buildings, roads, and bare land respectively;

[0029] Perform Softmax normalization on the scores of buildings, roads, and bare land output by the spatial classification model, convert the linear scores into probability distributions of buildings, roads, and bare land, and ensure that the sum of the probabilities of buildings, roads, and bare land is 1;

[0030] Adopt a weighted cross-entropy loss function, dynamically adjust the weights according to the class sample size, assign higher weights to small-sample classes, set the initial value of the learning rate to 0.001, adjust the learning rate in combination with the adaptive moment estimation optimizer, and add L2 regularization to constrain the convolution kernel parameters.

[0031] A further improvement of the technical solution of the present invention lies in: in the ground object classification module, when constructing a ground object classification model and outputting the fusion probability, the process includes:

[0032] The probability distributions of vegetation, soil, and water bodies output by the spectral classification model , and the probability distributions of buildings, roads, and bare land output by the spatial classification model are combined into a six-dimensional joint feature vector F, a gated fusion unit is introduced to dynamically calculate the contribution weights of spectral and spatial features , construct a ground object classification model, and output the fusion probability , and its calculation process is as follows:

[0033] ;

[0034] ;

[0035] Among them, is the Sigmoid function, and are trainable parameters, .

[0036] A further improvement of the technical solution of the present invention lies in: in the category arbitration module, when performing intelligent decision arbitration and outputting the final category, the process includes:

[0037] Set semantic compatibility rules. Vegetation and water bodies cannot coexist with buildings, and roads and bare land cannot coexist with water bodies. When the probability of any category in the fusion probability reaches 0.9, directly output the result. For areas where the fusion probability does not reach 0.9, if the output results of the spectral classification model and the spatial classification model conform to the semantic compatibility rules, select the category with the highest corresponding probability. If the output results of the spectral classification model and the spatial classification model conflict with the semantic compatibility rules, mark it as unknown, and the vegetation, soil, and water body categories are preferentially determined;

[0038] Calculate the maximum probability product of the spectral and spatial models through geometric mean as the comprehensive confidence level C, and its calculation process is as follows:

[0039] ;

[0040] If C reaches 0.85, mark it as high confidence and output the category with the highest probability. If C is between 0.6 and 0.85, the spectral category is dominant. If C is less than 0.6, it is an unknown category pixel, and secondary determination is preferentially based on the near-infrared reflectance and texture entropy value. If the near-infrared reflectance > 0.4, it is classified as vegetation. If the texture entropy value > 1.5, it is classified as a road. If it still cannot be classified, record the corresponding coordinates and trigger local re-acquisition by the drone;

[0041] According to the accuracy difference of the verification sets of the spectral and spatial models, dynamically adjust the fusion weight at a learning rate of 0.01, so that the high-precision ground object classification model occupies a higher decision-making weight in the arbitration.

[0042] A further improvement of the technical solution of the present invention lies in: in the category mapping module, the process of mapping the final category to the geographic information system and generating a thematic map of ground object classification includes:

[0043] Receive the final classification result and the corresponding pixel coordinates, and introduce affine transformation parameters to map the pixel coordinates to the geographic coordinate system. The final classification result includes vegetation, soil, water body, building, road, bare land, and unknown;

[0044] Generate a thematic map of ground object classification that annotates the associated category name, confidence level, acquisition timestamp, and conflict type, fill the thematic map of ground object classification with different colors corresponding to different categories, map the confidence level to transparency, overlay the classification result with the digital elevation model, and generate a three-dimensional ground object distribution map. The three-dimensional ground object distribution map is colored by elevation layer;

[0045] When the confidence level of the new data is higher than the old value, overwrite the original result, generate a time series change layer, calculate the change rate of the ground object area in adjacent periods, and mark the area where the change rate of the ground object area exceeds 5% with a flashing effect;

[0046] The thematic map of feature classification is stored in a tiled pyramid structure. The attribute data is indexed by the hash value of geographical coordinates, supporting the web map service protocol, returning raster tiles of specified areas and levels on demand, and providing a coordinate query interface to return the feature category and confidence level.

[0047] Generate an independent unknown layer for manual review. After triggering local re-sampling, automatically associate the old and new coordinates.

[0048] A further improvement of the technical solution of the present invention lies in: in the category mapping module, the process of presetting rules to export a classification statistical report includes:

[0049] Count the number of pixels by feature category, calculate the actual area in combination with the spatial resolution, compare the area change rate of adjacent periods, and automatically generate a classification statistical report including category area, area change rate of adjacent periods, and confidence level every day.

[0050] A further improvement of the technical solution of the present invention lies in: in the category mapping module, the process of setting the alarm threshold for the area change rate of feature categories and generating a feedback instruction includes:

[0051] Set the area thresholds for vegetation, soil, water body, building, road, bare land, and unknown. When the area change rate of adjacent periods exceeds the corresponding area threshold for 3 consecutive periods, trigger an alarm and mark the over-limit area.

[0052] If the proportion of the unknown area > 5% and the average confidence level < 0.7, generate a local re-sampling instruction, and the target area is the abnormal coordinate set. If the vegetation area decreases by more than 10% and the water body area decreases by more than 5%, it is determined as vegetation degradation and trigger an irrigation instruction. If the building and road areas increase by more than 5% and the bare land area increases by more than 15%, it is determined as building encroachment and trigger a law enforcement inspection instruction, marking the coordinates of the encroachment area.

[0053] Due to the adoption of the above technical solution, the technical progress achieved by the present invention compared with the prior art is:

[0054] 1. The present invention provides an automated feature classification system for multi-spectral and visible light images. By integrating multi-spectral and visible light data, it significantly improves the accuracy and efficiency of feature classification, providing strong support for the rapid acquisition and analysis of geospatial information.

[0055] 2. The present invention provides an automated feature classification system for multi-spectral and visible light images. Using the intelligent decision arbitration module, it effectively solves the problem of feature classification in complex scenarios, reduces classification errors, and improves the reliability and practicality of classification results.

[0056] 3. The present invention provides a multi-spectral and visible light image automatic ground object classification system. Through the category mapping module, it can automatically generate a ground object classification thematic map and a classification statistical report, and at the same time set an alarm threshold for the change rate of the ground object category area, providing convenient and efficient data support for geographical environment monitoring and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0058] Figure 1 It is a block diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0060] Embodiment 1, as Figure 1 shown, the present invention provides a multi-spectral and visible light image automatic ground object classification system, including a ground object data acquisition module, a ground object feature extraction module, a ground object classification module, a category arbitration module, and a category mapping module;

[0061] The ground object data acquisition module collects and preprocesses ground object multispectral data and visible light RGB image data. A multispectral imager and a visible light camera are carried on the UAV platform. Among them, the multispectral imager is equipped with near-infrared, red, green, and blue band sensors. The spectral resolution is less than 10 nanometers, and the spatial resolution is 10 meters. The visible light camera uses a three-channel charge-coupled device, with a spatial resolution of 0.5 meters and a frame rate of not less than 30 frames per second. It also has a built-in time synchronizer and a global positioning module. The multispectral imager divides the incident light according to the near-infrared, red, green, and blue bands. The detectors in each band synchronously record the reflected energy of the ground object to generate a radiance matrix, collect the ground object multispectral data, and through UAV flight path planning, make the adjacent image overlap rate higher than 30%. The visible light camera uses a Bayer filter to split light and separately records the light intensities in the red, green, and blue bands, outputting a three-channel digital matrix, collecting visible light RGB image data, performing radiometric correction and geometric correction on the collected ground object multispectral data and visible light RGB image data, using the dark current subtraction method to eliminate the influence of atmospheric scattering and compensate for the difference in solar altitude angle, establishing an affine transformation model based on ground control points to achieve coordinate registration, converting the multispectral data into a hierarchical data format, and converting the visible light image into a geotagged image format, and transmitting it to the ground object feature extraction module through a message queue;

[0062] The ground object feature extraction module extracts multispectral features and visible light features based on the preprocessed ground object multispectral data and visible light RGB image data. The multispectral features include vegetation index features and spectral reflection features, and the visible light features include spatial texture features and color distribution features. Based on the reflectance of the near-infrared and red bands of the multispectral data, the normalized difference vegetation index is obtained through its normalization to quantify the vegetation growth state and extract the vegetation index features. The mean reflectance of the green band is statistically calculated to characterize the vegetation activity, and the spectral fluctuation characteristics are described according to the standard deviation of the near-infrared band reflectance to construct a three-dimensional spectral feature vector and extract the spectral reflection features. The visible light RGB image is converted into a grayscale image, and the contrast and entropy are calculated using the gray-level co-occurrence matrix of the local window to extract the spatial texture features. The preprocessed visible light RGB image is converted to the HSV color space, and the hue peak is extracted to determine the main color. The discrete degree of the color distribution is described according to the standard deviation of the saturation to form a two-dimensional color feature vector and extract the color distribution features. The multispectral features and visible light features are respectively normalized by maximum-minimum normalization. The multispectral features are stored as a floating-point array, and the visible light features are encapsulated as a structured data packet. The bilinear interpolation method is used to unify the multispectral features and visible light features to a 0.5-meter grid. When the normalized difference vegetation index is greater than 1 and the hue peak is less than 0, abnormal pixels are automatically removed and the collected data is re-extracted;

[0063] The feature classification module combines multi - spectral features and visible - light features to construct a spectral classification model, a spatial classification model, and a feature classification model, and outputs the probability distributions of vegetation, soil, water, buildings, roads, and bare land, as well as the fusion probability. It integrates the normalized difference vegetation index, the mean value of the green - light band, and the standard deviation of the near - infrared band into a three - dimensional node feature vector. Using the prior probabilities of vegetation, soil, and water in the training samples as the initial prediction values, based on the gradient - boosting decision - tree algorithm, each decision tree uses the prediction residuals of the previous decision tree as the training target, selects the node feature that maximally reduces the Gini impurity for splitting, sets the maximum depth of each decision tree to 6 layers, and the minimum number of samples in the leaf nodes to 10 to construct the spectral classification model. The end - leaf nodes of each decision tree save the distribution weights of vegetation, soil, and water of the samples within the node, and the final output is the cumulative value of the weights of vegetation, soil, and water in 500 decision trees. The cumulative values of the weights of vegetation, soil, and water in 500 decision trees are converted into the probability distributions of vegetation, soil, and water through the Softmax function, and the sum of the probabilities of vegetation, soil, and water is 1. It combines the contrast generated during the process of extracting spatial texture features, the entropy, the hue peak generated during the process of extracting color distribution features, and the saturation standard deviation into a four - dimensional input feature vector, and constructs a spatial classification model based on the depth - separable convolution architecture. The spatial classification model independently performs spatial convolution on each input feature channel, extracts the spatial information of each input feature channel, fuses the features across channels through 1×1 point convolution to form a 32 - dimensional feature map, uses global average pooling to map the 32 - dimensional feature map into a global feature vector, compresses the global feature vector to 3 dimensions through a fully - connected layer, and each dimension corresponds to the scores of buildings, roads, and bare land respectively. It performs Softmax normalization on the scores of buildings, roads, and bare land output by the spatial classification model to convert the linear scores into the probability distributions of buildings, roads, and bare land, and the sum of the probabilities of buildings, roads, and bare land is 1. It adopts a weighted cross - entropy loss function, dynamically adjusts the weights according to the class sample size, assigns higher weights to small - sample classes, sets the initial value of the learning rate to 0.001, adjusts the learning rate in combination with the adaptive moment estimation optimizer, and adds L2 regularization to constrain the convolution - kernel parameters. The probability distributions of vegetation, soil, and water output by the spectral classification model , the probability distributions of buildings, roads, and bare land output by the spatial classification model are combined into a six - dimensional joint feature vector F, and a gated fusion unit is introduced to dynamically calculate the contribution weights of spectral and spatial features , and a feature classification model is constructed to output the fusion probability , and its calculation process is as follows:

[0064] ;

[0065] ;

[0066] where, is the Sigmoid function, and are trainable parameters, ;

[0067] The class arbitration module makes intelligent decision arbitration based on the probability distributions of vegetation, soil, water body, building, road and bare land, as well as the fusion probability, and outputs the final class. Set the semantic compatibility rules: vegetation and water body cannot coexist with buildings, and roads and bare land cannot coexist with water bodies. When the probability of any class in the fusion probability reaches 0.9, the result is directly output. For the area where the fusion probability is less than 0.9, if the output results of the spectral classification model and the spatial classification model conform to the semantic compatibility rules, the class with the highest corresponding probability is taken. If the output results of the spectral classification model and the spatial classification model conflict with the semantic compatibility rules, it is marked as unknown, and the vegetation, soil and water body classes are preferentially determined. The geometric mean is used to calculate the maximum probability product of the spectral and spatial models as the comprehensive confidence level C, and its calculation process is as follows:

[0068] ;

[0069] If C reaches 0.85, it is marked as high confidence and the class with the highest probability is output. If C is between 0.6 and 0.85, the spectral class is dominant. If C is less than 0.6, it is an unknown class pixel, and secondary determination is preferentially based on the near-infrared reflectance and texture entropy value. If the near-infrared reflectance > 0.4, it is classified as vegetation. If the texture entropy value > 1.5, it is classified as a road. If it still cannot be classified, the corresponding coordinates are recorded and the UAV is triggered for local re-sampling. According to the accuracy difference of the validation sets of the spectral and spatial models, the fusion weight is dynamically adjusted at a learning rate of 0.01, so that the high-precision ground object classification model occupies a higher decision weight in the arbitration;

[0070] The category mapping module maps the final categories to the Geographic Information System (GIS), generates a thematic map of feature classification, presets rules to export classification statistical reports, sets an alarm threshold for the area change rate of feature categories, and generates feedback instructions. It receives the final classification results and corresponding pixel coordinates, introduces affine transformation parameters to map the pixel coordinates to the geographic coordinate system. The final classification results include vegetation, soil, water body, building, road, bare land, and unknown. It generates a thematic map of feature classification that annotates the associated category names, confidence levels, acquisition timestamps, and conflict types, fills the thematic map of feature classification with different colors corresponding to different categories, maps the confidence level to transparency, overlays the classification results with the Digital Elevation Model (DEM) to generate a 3D feature distribution map, and colors the 3D feature distribution map by elevation stratification. When the confidence level of new data is higher than the old value, it overwrites the original result to generate a time-series change layer, calculates the area change rate of features in adjacent periods, and marks the areas where the area change rate of features exceeds 5% with a flashing effect. The thematic map of feature classification is stored in a tiled pyramid structure, and the attribute data is indexed by the hash value of the geographic coordinates, supporting the Web Map Service (WMS) protocol, returning raster tiles of specified regions and levels on demand, and providing a coordinate query interface to return the feature categories and confidence levels. It generates an independent unknown layer for manual review. After triggering local re-acquisition, it automatically associates the old and new coordinates, counts the number of pixels by feature category, calculates the actual area in combination with the spatial resolution, compares the area change rate in adjacent periods, and automatically generates a classification statistical report containing the category area, area change rate in adjacent periods, and confidence level every day. It sets area thresholds for vegetation, soil, water body, building, road, bare land, and unknown. When the area change rate in adjacent periods exceeds the corresponding area threshold for 3 consecutive periods, it triggers an alarm and marks the over-limit area. If the proportion of the unknown area is > 5% and the average confidence level is < 0.7, it generates a local re-acquisition instruction, and the target area is the set of abnormal coordinates. If the vegetation area decreases by more than 10% and the water body area decreases by more than 5%, it is determined that there is vegetation degradation and triggers an irrigation instruction. If the building and road areas increase by more than 5% and the bare land area increases by more than 15%, it is determined that there is building encroachment and triggers a law enforcement inspection instruction, marking the coordinates of the encroachment area.

[0071] First, the feature data acquisition module collects the multispectral data and visible light RGB image data of the required area, and performs preprocessing to improve the data quality. Subsequently, the feature extraction module extracts the multispectral features and visible light features from the preprocessed data. Then, the classification module constructs a classification model by integrating these features, and outputs the probability distributions of vegetation, soil, water, buildings, roads, and bare land, as well as the fusion probability. Next, the arbitration module makes an intelligent decision arbitration based on these probability distributions to determine the final feature category. Finally, the mapping module maps the final category to the geographic information system to generate a thematic map of feature classification, and exports a classification statistical report according to the preset rules. In addition, an alarm threshold for the change rate of feature categories can be set. When it is detected that the category change exceeds the threshold, the system will automatically generate a feedback instruction so as to take corresponding measures in a timely manner. The whole process realizes the integrated and automated processing from data acquisition to the application of classification results.

[0072] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A multi-spectral and visible light image automatic object classification system, characterized by: It includes a ground object data acquisition module, a ground object feature extraction module, a ground object classification module, a category arbitration module and a category mapping module; The ground object data acquisition module acquires and preprocesses ground object multispectral data and visible light RGB image data; The ground feature extraction module extracts multispectral features and visible light features based on the preprocessed ground feature multispectral data and visible light RGB image data; The object classification module integrates multi-spectral features and visible light features to build a spectral classification model, a spatial classification model and an object classification model, and outputs the probability distribution of vegetation, soil, water, buildings, roads and bare land, as well as the fusion probability; The category arbitration module performs intelligent decision arbitration based on the probability distribution of vegetation, soil, water, buildings, roads and bare land, as well as the fusion probability, and outputs the final category; The category mapping module maps the final category to the geographic information system, generates a feature classification thematic map, presets rules to derive classification statistical reports, sets a feature category area change rate alarm threshold, and generates feedback instructions.

2. The multi-spectral and visible light image automatic object classification system according to claim 1, characterized in that: In the ground object data acquisition module, the acquisition and preprocessing process of ground object multispectral data and visible light RGB image data includes: A multispectral imager and a visible light camera are installed on the UAV platform. The multispectral imager divides the incident light into near-infrared, red, green and blue bands. The detectors in each band synchronously record the reflected energy of the ground objects, generate a radiation brightness matrix, and collect multispectral data of the ground objects. The visible light camera uses a Bayer filter to split the light, respectively record the light intensity of the red, green and blue light bands, output a three-channel digital matrix, and collect visible light RGB image data; Radiation correction and geometric correction are performed on the collected multispectral data and visible light RGB image data. The dark current subtraction method is used to eliminate the influence of atmospheric scattering and compensate for the difference in solar altitude angle. An affine transformation model is established based on ground control points to achieve coordinate alignment. The multispectral data is converted into a hierarchical data format, and the visible light image is converted into a geographic tag image format, which are then transmitted to the feature extraction module via the message queue.

3. The multi-spectral and visible light image automatic object classification system according to claim 2, characterized in that: In the ground feature extraction module, the process of extracting multispectral features and visible light features includes: Multispectral features include vegetation index features and spectral reflectance features, and visible light features include spatial texture features and color distribution features; Based on the reflectance of near-infrared and red bands of multispectral data, the normalized vegetation index is obtained through its normalized difference, the vegetation growth state is quantified, and the vegetation index characteristics are extracted; The mean reflectance of the green light band is calculated to characterize the vegetation activity, the standard deviation of the reflectance of the near-infrared band is used to describe the spectral fluctuation characteristics, a three-dimensional spectral feature vector is constructed, and the spectral reflectance characteristics are extracted; Convert the visible light RGB image into grayscale image, calculate the contrast and entropy using the grayscale co-occurrence matrix of the local window, and extract the spatial texture features; Convert the preprocessed visible light RGB image to HSV color space, extract the hue peak to determine the main color, describe the discrete degree of color distribution according to the saturation standard deviation, form a two-dimensional color feature vector, and extract the color distribution characteristics; The multispectral features and visible light features are normalized to the maximum and minimum values ​​respectively. The multispectral features are stored as floating-point arrays, and the visible light features are encapsulated as structured data packets. The bilinear interpolation method is used to unify the multispectral features and visible light features into a 0.5-meter grid. When the normalized vegetation index is greater than 1 and the hue peak is less than 0, the abnormal pixels are automatically removed and the collected data is re-extracted.

4. The multi-spectral and visible light image automatic object classification system according to claim 3, characterized in that: In the object classification module, the process of constructing a spectral classification model and outputting the probability distribution of vegetation, soil and water bodies includes: The normalized difference vegetation index, the mean of the green light band and the standard deviation of the near infrared band are integrated into a three-dimensional node feature vector, and the prior probability of vegetation, soil and water in the training sample is used as the initial prediction value; Based on the gradient boosting decision tree algorithm, each decision tree takes the prediction residual of the previous decision tree as the training target, selects the node feature that makes the Gini impurity drop the most for splitting, sets the maximum depth of each decision tree to 6 layers, and the minimum number of leaf node samples to 10, to build a spectral classification model; The terminal leaf node of each decision tree stores the distribution weights of vegetation, soil and water bodies of the samples in the node. The final output is the cumulative value of the weights of vegetation, soil and water bodies in 500 decision trees. The cumulative value of the weights of vegetation, soil and water bodies in 500 decision trees is converted into the probability distribution of vegetation, soil and water bodies through the Softmax function, and the sum of the probabilities of vegetation, soil and water bodies is 1.

5. The multi-spectral and visible light image automatic object classification system according to claim 4, characterized in that: In the object classification module, the process of constructing a spatial classification model and outputting the probability distribution of buildings, roads and bare land includes: The contrast and entropy generated in the process of extracting spatial texture features and the hue peak and saturation standard deviation generated in the process of extracting color distribution features are combined into a four-dimensional input feature vector; Based on the deep separable convolutional architecture, a spatial classification model is constructed. The spatial classification model performs spatial convolution on each input feature channel independently, extracts the spatial information of each input feature channel, fuses features across channels through 1×1 point convolution, forms a 32-dimensional feature map, uses global average pooling to map the 32-dimensional features into a global feature vector, and compresses the global feature vector to 3 dimensions through a fully connected layer, where each dimension corresponds to the score of buildings, roads, and bare land. The scores of buildings, roads and bare land output by the spatial classification model are normalized by Softmax, and the linear scores are converted into probability distributions of buildings, roads and bare land, and the sum of the probabilities of buildings, roads and bare land is 1. The weighted cross entropy loss function is used to dynamically adjust the weights according to the number of category samples, giving higher weights to categories with small samples. The initial value of the learning rate is set to 0.001, and the learning rate is adjusted in combination with the adaptive moment estimation optimizer. L2 regularization is added to constrain the convolution kernel parameters.

6. The multi-spectral and visible light image automatic object classification system according to claim 5, characterized in that: In the object classification module, the process of constructing an object classification model and outputting a fusion probability includes: The probability distribution of vegetation, soil and water output by the spectral classification model , the probability distribution of buildings, roads and bare land output by the spatial classification model Merge into a six-dimensional joint feature vector F, introduce a gated fusion unit, and dynamically calculate the contribution weights of spectral and spatial features , build a ground feature classification model and output the fusion probability .

7. The multi-spectral and visible light image automatic object classification system according to claim 6, characterized in that: In the category arbitration module, the process of performing intelligent decision arbitration and outputting the final category includes: Set semantic compatibility rules: vegetation and water bodies cannot coexist with buildings, and roads and bare land cannot coexist with water bodies. When the probability of any category in the fusion probability reaches 0.9, the result is directly output. For areas where the fusion probability does not reach 0.9, if the output results of the spectral classification model and the spatial classification model meet the semantic compatibility rules, the category with the highest corresponding probability is selected. If the output results of the spectral classification model and the spatial classification model conflict with the semantic compatibility rules, they are marked as unknown, and the vegetation, soil and water categories are prioritized. The maximum probability product of the spectrum and the spatial model is calculated by geometric mean as the comprehensive confidence C. If C reaches 0.85, it is marked as high confidence and the highest probability category is output. If C is between 0.6 and 0.85, the spectral category is dominant. If C is less than 0.6, it is an unknown category pixel, and a secondary judgment based on near-infrared reflectivity and texture entropy value is given priority. If the near-infrared reflectivity is >0.4, it is classified as vegetation. If the texture entropy value is >1.5, it is classified as a road. If it still cannot be classified, the corresponding coordinates are recorded and the drone is triggered to re-sample locally. According to the difference in validation set accuracy between the spectral and spatial models, the fusion weight is dynamically adjusted with a learning rate of 0.01, so that the high-precision object classification model has a higher decision-making weight in arbitration.

8. The multi-spectral and visible light image automatic object classification system according to claim 7, characterized in that: In the category mapping module, the process of mapping the final category to the geographic information system and generating the feature classification thematic map includes: Receiving final classification results and corresponding pixel coordinates, introducing affine transformation parameters to map the pixel coordinates to a geographic coordinate system, wherein the final classification results include vegetation, soil, water, building, road, bare land, and unknown; Generate a feature classification thematic map with associated category name, confidence, acquisition timestamp and conflict type, fill the feature classification thematic map with different colors according to different categories, map the confidence to transparency, overlay the classification result with the digital elevation model, generate a three-dimensional feature distribution map, and color the three-dimensional feature distribution map according to elevation layers; When the confidence of the new data is higher than the old value, the original result is overwritten, and a time series change layer is generated. The area change rate of the adjacent period objects is calculated, and the area where the area change rate exceeds 5% is marked with a flashing effect; The feature classification thematic map is stored in a block pyramid structure, and the attribute data is indexed by the geographic coordinate hash value. It supports the network map service protocol, returns the raster slices of the specified area and level on demand, and provides a coordinate query interface to return the feature category and confidence level. Generate an independent unknown layer for manual review, and automatically associate the new and old coordinates after triggering local re-sampling.

9. The multi-spectral and visible light image automatic object classification system according to claim 8, characterized in that: In the category mapping module, the process of presetting rules to export classification statistical reports includes: The number of pixels is counted by feature category, the actual area is calculated based on the spatial resolution, and the area change rates of adjacent periods are compared. A classification statistical report containing category area, adjacent period area change rates and confidence levels is automatically generated every day.

10. The multi-spectral and visible light image automatic object classification system according to claim 9, characterized in that: In the category mapping module, the process of setting the alarm threshold of the area change rate of the ground feature category and generating a feedback instruction includes: Set area thresholds for vegetation, soil, water, buildings, roads, bare land and unknown areas. When the area change rate of adjacent cycles exceeds the corresponding area threshold for three consecutive cycles, an alarm is triggered and the exceeded area is marked. If the unknown area accounts for more than 5% and the confidence mean is less than 0.7, a local re-sampling instruction is generated, and the target area is the abnormal coordinate set. If the vegetation area decreases by more than 10% and the water area decreases by more than 5%, it is judged as vegetation degradation and the irrigation instruction is triggered. If the building and road area increases by more than 5% and the bare land area increases by more than 15%, it is judged as building encroachment, triggering the law enforcement inspection instruction and marking the coordinates of the encroached area.

Images