Image multi-modal feature extraction, ground feature classification and recognition and GIS image generation method

Abstract

The invention discloses an image multi-modal feature extraction method, a ground feature classification and recognition method and a GIS image generation method. Comprising the following steps: firstly, extracting texture features from a target image by adopting a multi-directional statistical method fusing rotation invariant coding of a local binary pattern and a gray-level co-occurrence matrix, and extracting color features from the target image by adopting an LAB-HSV dual-color space collaborative analysis method to obtain features of different modes of the target image; and then according to the information values of the extracted color features and texture features, adjusting the weight ratio corresponding to the color features and the texture features so as to optimize the recognition precision of the classification model on complex ground features. And finally, according to a weight ratio corresponding to the color feature and the texture feature, performing weighted fusion on the color feature and the texture feature to obtain a corresponding multi-modal feature vector.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for multimodal feature extraction, land cover classification and recognition, and GIS image generation. Background Technology

[0002] The main steps for obtaining GIS images from planned block geographic boundary maps (raw images) currently include: feature extraction, attribute assignment, and coordinate mapping. Feature extraction involves identifying and quantifying key geographic information from the raw image, such as land parcel boundaries, land use, and texture. This primarily includes using edge detection algorithms, such as the Canny operator, to identify feature boundaries in the image, providing a foundation for vectorization. It also involves using RGB threshold classification to extract color, texture, and other features to support land use classification decisions.

[0003] Attribute assignment involves associating extracted geographic features, such as land parcels, roads, and buildings, with their semantic information, such as usage, area, and ownership. Currently, attribute assignment is mainly achieved through manual interpretation of annotations (or rule-based matching).

[0004] Coordinate mapping transforms processed vector data, such as land parcel boundaries and road lines, from a pixel coordinate system to a real geographic coordinate system, giving the data spatial reference capabilities and supporting cross-platform spatial analysis and visualization. Coordinate mapping is typically achieved through affine transformations and projection transformations.

[0005] Due to the inherent defects of the edge detection algorithms currently used in the feature extraction step, the false negative rate of shadow areas in the original image and the confusion rate of plots of the same color are too high. The feature extraction method has low robustness and cannot adapt to the generation of GIS images in complex scenarios. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a method for image multimodal feature extraction, land cover classification and recognition, and GIS image generation, which can improve the robustness of feature extraction. The specific technical solution is as follows: In a first aspect, a method for extracting multimodal features from an image is provided. In a first possible implementation of the first aspect, it includes: Texture features are extracted from the target image using rotation-invariant coding and gray-level co-occurrence matrix, and the color features of the target image are analyzed collaboratively using LAB space and HSV space. Based on the information value corresponding to the color features and texture features, the weight ratios corresponding to the color features and texture features are dynamically adjusted. Based on the weight ratios corresponding to the color and texture features, the color and texture features corresponding to the target image are fused to obtain the multimodal feature vector corresponding to the target image.

[0007] In conjunction with the first possible implementation of the first aspect, in the second possible implementation of the first aspect, extracting the texture features and color features of the target image includes: The target image is enhanced using the multi-scale Retinex algorithm. Convert the enhanced target image to the LAB color space; Illumination compensation is performed on the luminance channel of the target image, while the original chroma information of the chroma channel of the target image is preserved; Calculate the luminance variance corresponding to the luminance channel of the target image, and compare the luminance variance with a set threshold. In response to the brightness variance being greater than a set threshold, the scale parameter of the multi-scale Retinex algorithm or the illumination compensation intensity of the brightness channel is adjusted, and the target image is reprocessed. In conjunction with the first possible implementation of the first aspect, in the third possible implementation of the first aspect, the extraction of texture features and color features of the target image includes: An edge detection algorithm is used to identify the contour edges of ground features in the target image; The target image is partitioned based on the contour edges of the ground features, and the filtering window size of different partitions is limited. The median filtering algorithm is then used to filter and denoise the target image.

[0008] In conjunction with the first possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the weight ratios corresponding to the color features and texture features are dynamically adjusted, including: Calculate the information entropy corresponding to the extracted color features and texture features, and assign weight ratios to the color features and texture features respectively according to the corresponding information entropy; Obtain environmental change indicators in the target image, and adjust the weight ratio of the color features and texture features according to the environmental change indicators; The weighted Euclidean distance between the target image and previously acquired historical images is calculated, and the change intensity index corresponding to the target image is calculated using an exponential smoothing algorithm. The average color saliency index of all historical images is calculated, and the weight ratio of the color feature and texture feature is adjusted by combining the color saliency index and change intensity index corresponding to the target image.

[0009] In conjunction with the first implementable method of the first aspect, the fifth implementable method of the first aspect employs rotation-invariant coding and gray-level co-occurrence matrix to extract texture features from the target image, including: An LBP coding matrix and a gray-level co-occurrence matrix are constructed based on the target image; The histogram feature vector of the target image is extracted using the LBP coding matrix, and the multidimensional directional feature vector of the target image is extracted using the gray-level co-occurrence matrix. The directional feature vector is fused with the histogram feature vector to generate the original feature pool, and the original feature pool is subjected to dimensionality reduction and redundant dimension filtering to obtain the texture features.

[0010] In conjunction with the first possible implementation of the first aspect, in the sixth possible implementation of the first aspect, the color features of the target image are analyzed collaboratively using LAB space and HSV space, including: The target image is converted into a LAB image and an HSV image respectively, and the LAB image and HSV image are aligned. The A-channel and B-channel images in the LAB image are divided into dense grids, and the main color clusters are identified by clustering based on the statistics of each grid cell to determine the structured chromaticity vector of the target image. Based on the saturation channel, the HSV image is divided into high-saturation and low-saturation regions. The hue histogram of the high-saturation region is calculated, and the low-saturation region is compensated by interpolation to determine the hue saliency vector of the target image. The HSV saturation weight is determined by the hue saliency vector, the chromaticity noise of the structured chromaticity vector is filtered by the HSV saturation weight, and the dynamic fusion coefficient is calculated based on the consistency of the main color distribution. Based on the dynamic fusion coefficient, the structured chromaticity vector and the hue salience vector are fused to generate a joint feature vector; The target image is divided into multiple sub-block images of the same size, and the joint feature vector is reduced in dimensionality. The color entropy and diversity index of each sub-block image are calculated to generate the color features of the target image.

[0011] Secondly, a method for classifying and identifying land features is provided. In a first implementable aspect of this second aspect, the method includes: The image multimodal feature extraction method as described in any one of claims 1-6 is used to extract multimodal feature vectors from the target image; Based on the multimodal feature vectors corresponding to the target image, the ground features in the target image are classified using a trained random forest model, and corresponding ground feature boundaries are generated.

[0012] In conjunction with the first feasible method of the second aspect, the corresponding feature boundaries are generated in the second feasible method of the second aspect, including: The random forest model is used to obtain the classification probability map corresponding to the target image, and multi-scale edge detection is performed based on the classification probability map to obtain the boundaries of different land features.

[0013] Thirdly, a GIS image generation method is provided, wherein in a first implementable manner of the third aspect, the method includes: The land feature classification and recognition method as described in claim 7 is used to classify and recognize the target image of the target area, and obtain the geographic boundary vector data corresponding to the target image. The geographic boundary vector data corresponding to the target image is mapped to a geographic coordinate system to generate a GIS image corresponding to the target area.

[0014] In conjunction with the first possible implementation of the third aspect, in the second possible implementation of the third aspect, mapping the geographic boundary vector data to a geographic coordinate system includes: At least four uniformly distributed ground control points are selected from the geographic boundary vector data, and a six-parameter transformation matrix is ​​fitted based on all the selected ground control points. The pixel coordinates in the geographic boundary vector data are converted into planar projection coordinates using the six-parameter transformation matrix, and the planar projection coordinates are converted into geographic coordinates using ellipsoidal projection.

[0015] Beneficial Effects: The image multimodal feature extraction, land cover classification and recognition, and GIS image generation method of this invention, by employing LAB and HSV spatial collaborative analysis of the color features of the target image, can solve the color distortion caused by the sensitivity of RGB space to brightness changes, as well as the problem of channel high correlation. This significantly improves the illumination robustness and color discrimination of the multimodal feature extraction method. Furthermore, by utilizing the geometric stability of rotation-invariant coding for edge direction features, the drastic changes in edge direction histograms caused by object rotation are resolved. Simultaneously, spatial constraint enhancement and multi-scale texture feature fusion through gray-level co-occurrence matrix address the problem that traditional rotation-invariant coding only considers local micro-modes and ignores the macro-structure of textures, improving texture discrimination and noise resistance. The complementary relationship between micro-rotation-invariant coding and macro-gray-level co-occurrence matrix improves the recognition accuracy for low-contrast textures, providing highly robust underlying feature support for dynamic weight allocation and ensuring the reliability of multimodal fusion in complex environments. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the specific embodiments will be briefly described below. In all the drawings, the elements or parts are not necessarily drawn to scale.

[0017] Figure 1This is a flowchart of an image multimodal feature extraction method provided in an embodiment of the present invention; Figure 2 A flowchart of a land cover classification and identification method provided in an embodiment of the present invention; Figure 3 This is a flowchart of a GIS image generation method provided in an embodiment of the present invention. Detailed Implementation

[0018] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0019] like Figure 1 The flowchart shown illustrates an image multimodal feature extraction method, which includes: Step 1: Extract texture features from the target image using rotation-invariant coding and gray-level co-occurrence matrix, and analyze the color features of the target image through collaborative analysis of LAB space and HSV space; Step 2: Dynamically adjust the weight ratios of the color features and texture features based on their corresponding information values; Step 3: Based on the weight ratios of the color features and texture features, fuse the color features and texture features of the target image to obtain the multimodal feature vector of the target image.

[0020] Specifically, firstly, a multi-directional statistical method combining rotation-invariant coding with local binary mode and gray-level co-occurrence matrix can be used to extract texture features from the target image. Then, a LAB-HSV dual-color space collaborative analysis method is employed to extract color features from the target image, obtaining features of different modalities. Next, based on the information value of the extracted color and texture features, the weight ratios of the corresponding color and texture features can be adjusted to optimize the classification model's accuracy in recognizing complex features. Finally, the extracted color and texture features can be weighted and fused according to their respective weight ratios to obtain a multi-modal feature vector of the target image.

[0021] In this embodiment, optionally, the extraction of texture and color features of the target image includes: Evaluate the noise level and illumination uniformity of each target image; The target image is subjected to illuminance normalization and noise filtering based on the illuminance uniformity and noise level, respectively.

[0022] Specifically, after acquiring geospatial images, a seven-parameter coordinate transformation model can be used to transform geospatial images in local coordinate systems to global coordinate systems. Existing spatial transformation tools can then be used to align the coordinates, ensuring geospatial consistency and thus achieving data standardization.

[0023] Specifically, the seven-parameter coordinate transformation model includes three translation parameters. 3 rotation parameters and scale factor The specific calculation formula for converting local coordinates to global coordinates using the seven-parameter coordinate transformation model is as follows: ; in, For local coordinates, The rotation matrix is ​​expressed as follows: .

[0024] To accelerate image processing, the geospatial image can be segmented into multiple target images according to a set block scale, allowing for parallel processing of these images. When processing the target images, existing noise level and illumination uniformity assessment methods can be used to evaluate the noise level and illumination uniformity of the central region of the target image, respectively, and these evaluations can be used as the corresponding noise level and illumination uniformity of the target image.

[0025] In this embodiment, the noise level can be assessed by the mean absolute difference between the target image and the Gaussian filter result. The specific calculation formula is as follows: ; in, The pixel values ​​of the target image. The result of Gaussian filtering of the target image. Indicates image resolution.

[0026] Maximum value of the LAB luminance channel of the target image and minimum value The difference between the values ​​is used to assess the uniformity of illumination. The specific calculation formula is as follows: .

[0027] Finally, the noise level and illumination uniformity of the target image can be compared with the pre-set corresponding noise level thresholds and illumination uniformity thresholds, and appropriate processing can be performed based on the comparison results. The relationship between the comparison results and the processing methods is shown below: Noise assessment Illumination uniformity assessment In this embodiment, optionally, the target image undergoes illuminance normalization and noise filtering processing, including: The target image is enhanced using the multi-scale Retinex algorithm.

[0028] Specifically, to eliminate uneven lighting in images, such as uneven lighting distribution caused by shadows, reflections, and low brightness, the target image can first be enhanced using a multi-scale Retinex algorithm to suppress lighting interference at different scales, thereby enhancing the detail recovery capability in shadow areas. Then, LAB spatial transformation is used to adjust the images under different lighting conditions to a uniform brightness range, improving the stability of subsequent feature extraction and classification, and enhancing the robustness of feature extraction.

[0029] Specifically, the actual kernel size of the Gaussian kernel can first be calculated based on the scale parameter. The specific calculation formula is as follows: ; in, This is the initial scale.

[0030] Then, the MSR algorithm is executed in parallel on the RGB three channels to calculate the reflection components. The specific calculation formula is as follows: ; in, It is a Gaussian filter kernel. These are the weighting coefficients corresponding to different scales.

[0031] During the calculation, shadow areas with a brightness less than 50 can be identified based on the brightness distribution in the target image, and the medium scale can be increased. The weighting coefficients are adjusted dynamically. By dynamically adjusting the weighting coefficients of the scale and using multi-scale fusion, the ability to restore details in shadow areas can be enhanced, while avoiding overexposure.

[0032] In this embodiment, optionally, the illuminance of the target image is normalized, including: Convert the enhanced target image to the LAB color space; Illumination compensation is performed on the luminance channel of the target image, while the original chroma information of the chroma channel of the target image is preserved; Calculate the luminance variance corresponding to the luminance channel of the target image, and compare the luminance variance with a set threshold. In response to the brightness variance being greater than a set threshold, the scale parameter of the multi-scale Retinex algorithm or the illumination compensation intensity of the brightness channel is adjusted, and the target image is reprocessed.

[0033] Specifically, firstly, the target image after image enhancement can be converted to the LAB color space to separate the luminance and chrominance channels, thus eliminating chrominance interference. Then, existing illumination compensation methods, such as histogram equalization or adaptive stretching, can be used to compensate the luminance channel of the target image, while preserving the original chrominance of the chrominance channel to avoid color distortion. Next, the luminance variance of the compensated luminance channel can be calculated and compared with a set luminance variance threshold to confirm whether the normalization result meets the requirements. If it does not meet the requirements, the scale parameters of the multi-scale Retinex algorithm can be adjusted, such as increasing the weight coefficient of the mesoscale Gaussian kernel. Alternatively, the illumination compensation intensity of the luminance channel can be adjusted, and the target image can be normalized again to provide consistent image input for subsequent noise suppression and feature extraction. The specific formula for calculating the luminance variance is as follows: ; in, For pixels The corresponding brightness value, The average brightness of the target image. , These represent the number of pixels horizontally and vertically in the target image, respectively.

[0034] In this embodiment, optionally, noise filtering processing is performed on the target image, including: The filter window size is dynamically selected based on the noise level of the target image, and an adaptive median filtering algorithm is used to filter and denoise the target image.

[0035] Specifically, to eliminate sensor noise, scanning artifacts, and environmental interference while preserving ground feature edge details and providing high signal-to-noise ratio data for subsequent feature extraction, the filter window size can be dynamically selected based on the noise level of the target image, and an adaptive median filtering algorithm can be used to filter and denoise the target image.

[0036] Specifically, the noise level of the target image can first be calculated using the aforementioned noise level algorithm. When the noise level... When the noise level is low, the filter window size can be 3. At this point, the noise level is considered medium, and the filter window size can be 5. When the noise level... When the noise level is high, the filter window size can be 7.

[0037] After setting the filter window size according to the noise level of the target image, the median filtering algorithm can be used to filter the target image. The specific calculation formula is as follows: ; in, Where is the filter radius, and the filter window size is = .

[0038] During the filtering process, GPUs can be used to perform parallel computations on multiple target images to improve image processing efficiency.

[0039] In this embodiment, optionally, an adaptive median filtering algorithm is used to filter and denoise the target image, including: An edge detection algorithm is used to identify the contour edges of ground features in the target image; The target image is partitioned based on the outline edges of the ground features, and the filter window size of different partitions is limited.

[0040] Specifically, while median filtering alone can reduce noise in geographic images, it may result in blurred boundaries. Therefore, edge protection strategies can be employed during the filtering process to identify true edge regions and limit the filtering intensity, preventing blurring of feature outlines. Specifically: First, the Canny edge detection algorithm is used to detect the contour edges of ground features in the target image. During the detection process, existing sub-pixel edge localization can be used for correction to achieve a localization accuracy of 0.5 pixels. The specific calculation formula for sub-pixel edge localization is as follows: ; in, , For the Sobel gradient of a pixel, , Let be the second derivative of the Sobel gradient of the pixel.

[0041] Then, based on the outline edges of ground features, the target image can be divided into edge regions, non-edge regions, and buffer zones. The edge region is the area covered by the outline edges of ground features, the buffer zone is the area extending outwards by two pixels from the edge region, and the remaining areas are non-edge regions. For the edge region, the filtering radius of the filtering window is limited. For non-edge regions, the filtering radius of the filtering window is set according to the noise level. For buffer zones, a gradual window size strategy is used, meaning the filtering radius of the filtering window gradually increases from the inside out. For example, it can be set to... This avoids mutation artifacts.

[0042] After filtering and denoising the target image, the degree of information loss during noise suppression or reconstruction can be assessed based on the mean square error between the processed target image and the original image. If the degree of information loss meets the requirements, subsequent calculations can be performed; otherwise, the target image needs to be reprocessed.

[0043] In this embodiment, optionally, dynamically adjusting the weight ratios corresponding to the color features and texture features includes: The weighted Euclidean distance between the target image and previously acquired historical images is calculated, and the change intensity index corresponding to the target image is calculated using an exponential smoothing algorithm. The average color saliency index of all historical images is calculated, and the weight ratio of the color feature and texture feature is adjusted by combining the color saliency index and change intensity index corresponding to the target image.

[0044] Specifically, firstly, multimodal feature streams, such as color and texture features, from historical images of the target region can be obtained. A fixed-length (default 5 frames) circular buffer is then created, and PCA dimensionality reduction (64-dimensional → 16-dimensional) is performed on the texture and color features of each frame of the target image before storage. This generates a multi-frame historical feature set, providing a foundational data pool for change analysis.

[0045] Then, the weighted Euclidean distance between the current target image and historical images of the target region can be calculated. An exponential smoothing formula is then applied to generate the intensity of change; a higher intensity of change indicates a more stable scene.

[0046] Then, it can be based on the change intensity value. Adjust the static edge weights; the lower the intensity of change (in drastic change scenarios), the larger the weight increase. Hard constraints are set in the [0.2, 0.8] range. To prevent noise from causing weights to become uncontrollable, the weights can be limited to a fixed area. Output adaptive edge weights to enhance the dominance of the geometry in dynamic environments.

[0047] Finally, the color weights can be scaled based on the ratio of the current target image's color saliency to the historical mean of the historical feature set, combined with the change intensity value. This applies when the scene is stable (i.e.,...). ) Enhance the effect of color, especially during dramatic scene changes (i.e. The color weights are attenuated. The texture weights are adjusted synchronously and inversely, thereby achieving bimodal adaptive complementarity.

[0048] In this embodiment, optionally, rotation-invariant coding and gray-level co-occurrence matrix are used to extract texture features from the target image, including: Construct an LBP coding matrix and a gray-level co-occurrence matrix based on the target image; The histogram feature vector of the target image is extracted using the LBP coding matrix, and the multidimensional directional feature vector of the target image is extracted using the gray-level co-occurrence matrix. The directional feature vector is fused with the histogram feature vector to generate the original feature pool, and the original feature pool is subjected to dimensionality reduction and redundant dimension filtering to obtain the texture features.

[0049] Specifically, firstly, the LBP encoding matrix and gray-level co-occurrence matrix of the target image can be constructed. Specifically, a grayscale image of the target image (LAB luminance channel) is obtained first. Then, circular neighborhood sampling (radius 2 pixels, 8 neighboring points) is used. Uniform regions are excluded by contrast threshold filtering (>10 gray levels). Finally, the binary relationship between neighboring pixels and the center point is calculated, and the minimum value is taken as the feature value through cyclic shifting. The LBP encoding matrix is ​​generated by combining the binary relationship between neighboring pixels and the center point.

[0050] Using an LBP encoding matrix to extract texture features can eliminate the effects of rotation, thus better representing the shape features of objects in the image, unaffected by pose changes. It also compresses feature dimensions and reduces biases caused by image lighting, brightness, position, and size. Furthermore, contrast threshold filtering (>10 gray levels) ensures that only valid textures are retained.

[0051] When constructing the gray-level co-occurrence matrix, the four orientation parameters and distance parameters are first rotated. Then, the grayscale image is traversed according to the orientation and distance parameters, and the frequency of occurrence of pixel pairs that satisfy the orientation and distance parameters is counted, i.e., the joint probability of pixel pairs. Based on the joint probability of pixel pairs, the gray-level co-occurrence matrix is ​​generated according to the grayscale level of the grayscale image.

[0052] The image can then be segmented into 16×16 sub-blocks, and a 256-bin uniform pattern histogram for each sub-block is calculated using an LBP encoding matrix. The calculated local histograms are then normalized and sequentially concatenated to form the histogram feature vector of the target image. During histogram calculation, a sliding window (8-pixel stride) can be used to achieve 50% overlap sampling, enhancing spatial continuity. The calculated histogram feature vector can characterize the differences in texture distribution across different regions.

[0053] Five statistical measures in different directions can be calculated based on the constructed gray-level co-occurrence matrix: contrast (texture roughness), correlation (linear dependence), energy (uniformity), homogeneity (local uniformity), and entropy (randomness), resulting in a total of 20-dimensional directional feature vectors. Finally, the concatenated histogram feature vectors and orientation feature vectors can be combined to form the original feature pool. PCA is then performed on the original feature pool to reduce dimensionality, preserving 95% of the variance information and compressing the dimension to 64. Redundant dimensions are then filtered out using correlation coefficient analysis (>0.7), outputting the optimized fused texture features. Furthermore, the difference between local texture and global distribution can be compared using chi-square distance to generate a saliency heatmap. Feature attenuation is applied to the fused texture features in low-saliency regions (<0.1) to highlight key texture structures. Finally, L2 normalization is performed to output standardized texture features.

[0054] In this embodiment, optionally, the color features of the target image are analyzed collaboratively using LAB and HSV color spaces, including: The target image is converted into a LAB image and an HSV image respectively, and the LAB image and HSV image are aligned. The A-channel and B-channel images in the LAB image are divided into dense grids, and the main color clusters are identified by clustering based on the statistics of each grid cell to determine the structured chromaticity vector of the target image. Based on the saturation channel, the HSV image is divided into high-saturation and low-saturation regions. The hue histogram of the high-saturation region is calculated, and the low-saturation region is compensated by interpolation to determine the hue saliency vector of the target image. The HSV saturation weight is determined by the hue saliency vector, the chromaticity noise of the structured chromaticity vector is filtered by the HSV saturation weight, and the dynamic fusion coefficient is calculated based on the consistency of the main color distribution. Based on the dynamic fusion coefficient, the structured chromaticity vector and the hue salience vector are fused to generate a joint feature vector; The target image is divided into multiple sub-block images of the same size, and the joint feature vector is reduced in dimensionality. The color entropy and diversity index of each sub-block image are calculated to generate the color features of the target image.

[0055] Specifically, firstly, the LAB image (L-channel brightness equalization) after illumination normalization can be spatially aligned with the original RGB-converted HSV image. Affine transformation is then used to ensure strict matching of the pixel coordinates in the two spaces, laying the foundation for subsequent collaborative analysis.

[0056] Then, chroma channel analysis can be performed in LAB space, dividing the A / B channels into a dense grid and calculating the statistics (mean / variance) of each cell. Furthermore, dominant color clusters are identified through clustering, generating elliptical parameters (center, axis, range) that characterize the color distribution features, resulting in a structured chroma vector for the target image.

[0057] Next, leveraging the perceptual advantages of the HSV color space, high / low saturation regions can be segmented based on saturation channels. Then, hue histograms are calculated only in the high saturation regions, and interpolation compensation is performed on the low saturation regions to enhance the effectiveness of hue features. Finally, the hue saliency vector of the target image is obtained.

[0058] Then, a LAB-HSV feature association mechanism is established, using HSV saturation weights to filter chromatic noise in LABs, and a dynamic fusion coefficient is calculated based on the consistency of the main color distribution. The specific calculation formula is as follows: ; in, Number of primary color clusters As the primary color center of the LAB color space, As the primary color center of the HSV color space, Euclidean distance can be represented by the chromatic distance in the LAB color space. This represents a color space mapping function. When calculating dynamic blending coefficients, the color space mapping function can be used to map the primary color center of the HSV color space. Switch to LAB space. This is the color difference tolerance coefficient, which can be adaptively adjusted based on the average saturation. The specific adjustment method is as follows: .

[0059] In this embodiment, the HSV saturation weights can be determined through saliency analysis and spatial probability modulus of the HSV image. Specifically, the histogram of the S-channel image of the HSV image is first calculated, and the segmentation threshold is obtained using the Otsu's method. Then, morphological closing operations are performed on the binary mask to eliminate noise points and connect broken regions in the HSV image. Next, the spatial weights and local color weights of the HSV image are calculated separately, and the spatial weights and local color weights are combined to form the initial weights. Finally, adaptive calibration can be performed on the shadow areas and overexposed areas according to the set calibration coefficients to obtain the final HSV saturation weights.

[0060] Subsequently, a joint feature vector can be generated by fusing the hue saliency vector and structured chromaticity vector of the target image using a dynamic fusion system. Weights can be adaptively allocated based on lighting changes (LAB is dominant in sunny conditions, while HSV is enhanced in foggy conditions). For shadow / overexposure issues, the chromaticity histogram can be stretched in low-brightness areas to restore true colors. Furthermore, the HSV luminance channel is used to correct color casts in shadow areas. This significantly improves chromaticity stability in over 90% of scenes with lighting interference.

[0061] Finally, the target image can be divided into multiple sub-images of the same size, and the joint feature vector can be reduced in dimensionality. Then, the color entropy and diversity index of each sub-image can be calculated based on the joint feature vector. Finally, color features containing global feature vector, spatial distribution vector and primary color metadata are obtained to support subsequent high-precision classification.

[0062] In this embodiment, optionally, fusing the color features and texture features corresponding to the target image includes: Principal component analysis (PCA) is used to optimize the dimensionality of the extracted high-dimensional features. Specifically, after extracting color and texture features, PCA can be used to optimize the dimensionality of the high-dimensional features, retaining 95% of the variance information to compress redundant data, thereby effectively improving the robustness of feature extraction.

[0063] In this embodiment, optionally, dynamically adjusting the weight ratios corresponding to the color features and texture features includes: Information entropy analysis is performed on the extracted color and texture features, and the weight ratios of the color and texture features are adjusted according to their information entropy.

[0064] Specifically, information entropy analysis can be performed on the extracted color and texture features to quantify the discriminative power of each feature dimension in the sample set. In this embodiment, information entropy can be calculated by statistically analyzing the probability distribution of feature values; the lower the entropy value, the greater the contribution of the feature to class differentiation. For example, color features in high-contrast regions typically have low entropy values, while texture features in homogenized regions may have higher entropy values.

[0065] Assigning weights to color and texture features based on information entropy calculation results allows high-discriminative features to receive higher weights. For example, a specific color combination of metal rooftops in an industrial site might be assigned a weight of 0.6 due to its low entropy value, while texture features affected by shadows might have a lower weight of 0.2 due to their higher entropy value. To ensure the stability of weight allocation, hard constraints can be set. For instance, a color feature weight could be mandated to be no less than 0.35 to prevent texture noise from dominating classification results in low-light or blurred scenes.

[0066] Furthermore, it can monitor environmental change indicators in real time. For example, if the average saturation of the image is detected to be below 30, it is determined to be a cloudy day or low light conditions. Based on the environmental change indicators, a dynamic weight adjustment mechanism is triggered. For example, when the shadow coverage area exceeds 20%, the texture feature weight is temporarily increased to 0.5 to compensate for the loss of color information, while maintaining the lower limit constraint of color weight.

[0067] Furthermore, a sliding window algorithm can be introduced to smooth the weights. Specifically, the median entropy value of 10 consecutive samples can be used as a benchmark to avoid weight jumps caused by anomalies in a single sample. For example, when processing historical atlases, if fluctuations in color entropy values ​​are detected in multiple consecutive blocks due to yellowing of paper, a sliding window correction can be automatically activated to compress the weight fluctuation range from ±0.15 to ±0.05, significantly improving cross-regional classification consistency.

[0068] Thus, by integrating dynamic evaluation of feature information entropy, rule constraints of scene perception, and data-driven smooth optimization, color and texture features can complement each other in different geographical environments, providing highly robust and discriminative feature inputs for subsequent classification, supporting accurate land parcel division and usage identification in complex scenarios, and improving the robustness of multimodal feature extraction methods.

[0069] The technical effects of the present invention were verified through experiments, and the experimental results are shown in the table below: Therefore, the multimodal feature extraction method of this application can significantly improve the accuracy and robustness of feature extraction.

[0070] like Figure 2 The flowchart shown illustrates a method for classifying and identifying land features. This method includes: Step S1: Extract multimodal feature vectors from the target image using the above-described image multimodal feature extraction method; Step S2: Based on the multimodal feature vectors corresponding to the target image, classify the ground features in the target image using a trained random forest model and generate corresponding ground feature boundaries.

[0071] Specifically, firstly, the multimodal feature extraction method described above can be used to extract multimodal feature vectors from the target image. Then, the multimodal feature vectors are input into a pre-trained random forest model. The random forest model can classify the land features in the target image, determine the category and boundaries of the land features, and obtain the geographic boundary vector data corresponding to the target image.

[0072] The random forest model consists of multiple decision trees, with each tree limited to a maximum depth of 15 layers to prevent overfitting. Gini impurity is used as the node splitting criterion to ensure that highly confusing categories, such as residential land versus commercial land, are prioritized during feature selection. During single-tree construction, each node randomly selects only 8 features for splitting and evaluation; this feature subset sampling strategy enhances the model's generalization ability.

[0073] When classifying, a majority voting mechanism can be used to count the prediction results of all decision trees for the current sample. When the highest number of votes exceeds 85%, the category label (such as R1 residential land) is directly output. Otherwise, a manual review process is triggered and the spatial location of low confidence samples is recorded.

[0074] In the boundary generation stage, multi-scale edge detection can be performed based on the classification probability map. Specifically, Gaussian kernels with scales σ=1.6, 2.4, and 3.2 are used to generate edge responses at different levels of detail. Non-maximum suppression methods are used to preserve local extrema along the gradient direction, eliminating artifact interference. For continuous edge point sets, the Douglas-Peucker algorithm can be used for polygon simplification.

[0075] Specifically, first set the simplification threshold ε to 0.5% of the original perimeter (e.g., for a boundary with a perimeter of 2000 pixels, the maximum simplification error is allowed to be 10 pixels), and ensure that the intersection-union ratio (IoU) between the simplified vector boundary and the original raster classification result is not less than 0.9.

[0076] Then, the polygon formed based on the continuous set of edge points is used as the original polygon, and the start and end points of the original polygon are marked as points that must be retained and included in the result set. Then, a segmented processing stack is created, and the entire curve (start point...) is processed... To the finish line The initial segment is pushed onto the stack.

[0077] Then iterate through all vertices in the segment being processed. Calculate the line segment from the vertex to the start point and the end point. Find the vertical distance and record the vertex corresponding to the maximum vertical distance. and distance value The specific formula for calculating the vertical distance is as follows: ; in, , , These are the image coordinates of the start point, end point, and vertex, respectively.

[0078] If distance value If the distance value is less than or equal to a preset threshold, all intermediate points of that segment are discarded. If the value exceeds a preset threshold, then the vertex will be... Include it in the result set and split the current segment into two sub-segments, the left segment being... The right section is Then push the two sub-segments onto the processing stack and continue the calculation. Repeat this process until the segment only includes the end point and the start point.

[0079] Then, all result points in the result set are rearranged in their original order to maintain geometric continuity. A topological check is performed to verify that the simplified polygons are self-intersecting; if intersections are found, the threshold is reduced to 80%, and the algorithm is re-executed. Finally, the optimized polygon is output, returning the simplified point sequence, and the compression rate is recorded synchronously (e.g., from 200 points to 60 points, the compression rate is 70%).

[0080] For complex terrains (such as jagged coastlines), an adaptive segmentation simplification strategy can be used to retain more vertices in segments with curvature changes exceeding 15 degrees. The final output geographic boundary vector data includes land use classification labels, boundary coordinate sequences, and confidence metadata.

[0081] Specifically, curvature analysis is first performed on the vertices of the original polygon to calculate the turning angle of adjacent line segments. Based on the angle values, the curve is divided into three types of regions: flat regions (turning angle less than 15 degrees), medium curvature regions (15-45 degrees), and critical turning regions (greater than 45 degrees). Continuous high curvature segments are marked as key feature areas, and a curvature heatmap is generated to guide subsequent simplification. Then, a base threshold is calculated based on the initial perimeter of the original polygon, and adaptively adjusted according to different curvature regions. Specifically, a 1.5x loose threshold is used in flat regions to improve the compression ratio, the base threshold is maintained in medium-curvature regions to maintain balance, and a 0.3x strict threshold is applied to key transition regions to ensure the integrity of transition features. A gradual transition zone is set at the boundaries of regions to achieve smooth threshold transitions.

[0082] Then, based on the curvature partitioning results, the original polygon is divided into multiple sub-segments. The Douglas-Peucker simplification algorithm is executed independently on each sub-segment. Flat segments use a relaxed threshold to significantly compress redundant points, critical segments use a strict threshold to retain important inflection points, and medium segments maintain a balanced strategy. Different curvature regions receive differentiated simplification processing.

[0083] Then, special processing is applied to the connection points at the segment boundaries. The distance between adjacent segment endpoints is detected, and if the spacing exceeds the standard, an intermediate point is inserted to restore geometric continuity. Next, the deviation between the actual turning angle of the connection point and the original angle is calculated. If it exceeds the threshold, auxiliary points are added to correct the turning features, ensuring that the segment connection is natural and smooth. Finally, Hausdorff distance is used to evaluate the overall simplification accuracy, with a focus on monitoring local deviations in key areas. When the error in a key section exceeds the limit, the threshold for that area is automatically lowered for re-simplification. For flat sections with insufficient compression, the threshold is increased for secondary optimization. Through a closed-loop feedback mechanism, compression efficiency is maximized while maintaining geometric accuracy.

[0084] For cross-block regions in a target image (such as roads spanning multiple target images), topological verification can be performed to eliminate coordinate gaps or overlaps at seams. Through the deep fusion of machine learning classification and geometric processing, automated mapping from pixel-level features to geographic vector edges is achieved, providing standardized data products that can be directly applied to spatial planning and land management.

[0085] Specifically, firstly, the shared boundary lines of adjacent blocks are scanned, and corresponding points are quickly matched using spatial indexing. By measuring the point spacing and analyzing boundary intersections, two main types of problems are automatically identified: one is the gap problem (the discontinuity between adjacent features exceeds the allowable value), and the other is the overlap problem (redundant overlapping areas are generated by features in different blocks). The system automatically marks the problem areas and classifies and archives them, providing a target list for subsequent repairs.

[0086] Then, differentiated repair schemes are initiated for different problem types. For fractured areas, a triangular mesh is established to connect breakpoints, and original image features within the buffer zone are extracted to guide the repair. For overlapping areas, the optimal dividing line is determined by radial scanning after calculating the centroid of the intersection. Structured features such as roads are connected using geometric extension lines, while unstructured areas such as water bodies and green spaces are filled using morphological closing operations.

[0087] After geometric restoration, semantic consistency is verified. Specifically, attribute labels (such as road names and land use types) of the same feature within adjacent blocks are compared, and a pre-trained language model is used to analyze text similarity. For ambiguous or contradictory attribute values, intelligent arbitration is performed based on confidence weights and spatial location to ensure that key attributes such as names and classifications are completely consistent across blocks.

[0088] Then, the repair results are verified using two levels of accuracy. Specifically, the geometric layer evaluates the repair effect using Hausdorff distance, requiring a deviation of less than 0.5 pixels; the attribute layer checks the consistency of label propagation. A closed-loop feedback mechanism is set up to automatically expand the repair range and iterate through unqualified areas. If three consecutive failures occur, manual intervention is initiated, and a detailed error report is generated. Finally, key parameters of successful repair cases (such as crack threshold and morphological kernel size) can be stored in a topology rule knowledge base. By analyzing historical repair data through machine learning, the repair strategy parameters for each scenario are automatically optimized, and the system continuously evolves to improve its ability to handle complex situations.

[0089] like Figure 3 The flowchart shown illustrates a GIS image generation method, which includes: Step D1: Use the above-mentioned land feature classification and recognition method to classify and recognize the target image of the target area, and obtain the geographic boundary vector data corresponding to the target image; Step D2: Map the geographic boundary vector data corresponding to the target image to the geographic coordinate system to generate a GIS image corresponding to the target area.

[0090] Specifically, firstly, the aforementioned land feature classification and recognition methods can be used to identify the land feature categories and boundaries in the target image, obtaining the corresponding geographic boundary vector data. Then, the geographic boundary vector data can be mapped to a geographic coordinate system, achieving strict alignment between spatial data and actual geographic location, thereby generating a GIS image of the target area.

[0091] In this embodiment, optionally, mapping the geographic boundary vector data to a geographic coordinate system includes: At least four uniformly distributed ground control points are selected from the geographic boundary vector data, and a six-parameter transformation matrix is ​​fitted based on all the selected ground control points. The pixel coordinates in the geographic boundary vector data are converted into planar projection coordinates using the six-parameter transformation matrix, and the planar projection coordinates are converted into geographic coordinates using ellipsoidal projection.

[0092] Specifically, first, the mathematical relationship between pixel coordinates and planar projection coordinates can be established through affine transformation. Specifically, at least 4 evenly distributed ground control points (such as road intersections, building corners) can be selected from the geographical boundary vector data, and the six-parameter transformation matrix (including translation, rotation, and scaling factors) can be fitted using the least squares method. During the fitting process, it is required that the control point fitting residual (RMSE) is less than 0.5 pixels to ensure geometric accuracy. Then, using the six-parameter transformation matrix obtained by fitting, all pixel coordinates in the geographical boundary vector data are converted into planar projection coordinates.

[0093] After completing the conversion of planar projection coordinates, the planar projection coordinates can be converted into WGS84 longitude and latitude through ellipsoidal projection. Considering the influence of the earth's curvature and elevation, the CGCS2000 ellipsoid parameters (semi-major axis a = 6378137 meters, flattening f = 1 / 298.257222101) can be used for three-dimensional space calculation. For high-altitude areas (such as mountainous areas), DEM elevation data needs to be introduced to correct the projection deviation. To ensure data integrity, a coordinate back-substitution test can be performed, that is, 10% of the boundary points are randomly selected and mapped back to pixel coordinates, and the back-substitution error between the pixel coordinates obtained by reverse mapping and the true pixel coordinates in the geographical boundary vector data is calculated. If the back-substitution error is less than 1 pixel, it is qualified. Otherwise, the number of control points is increased and the six-parameter transformation matrix is refitted, and coordinate conversion is performed through the refitted six-parameter transformation matrix.

[0094] The finally output vector data in GeoJSON or Shapefile format contains WGS84 longitude and latitude coordinates, attribute labels, and accuracy metadata. By stitching together the GIS images of all target images corresponding to the target area, the GIS image of the target area can be obtained, and cracks or overlaps generated by cross-target image block stitching can be eliminated through topological inspection. For example, the seam error of a cross-map road boundary is less than 0.1 meters after topological optimization.

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present invention, and they should all be covered within the scope of the claims and the description of the present invention.

Claims

1. A method for extracting multimodal features from an image, characterized in that, include: Texture features are extracted from the target image using rotation-invariant coding and gray-level co-occurrence matrix, and the color features of the target image are analyzed collaboratively using LAB space and HSV space. Based on the information value corresponding to the color features and texture features, the weight ratios corresponding to the color features and texture features are dynamically adjusted. Based on the weight ratios corresponding to the color and texture features, the color and texture features corresponding to the target image are fused to obtain the multimodal feature vector corresponding to the target image.

2. The image multimodal feature extraction method according to claim 1, characterized in that, Extracting the texture and color features of the target image, including: The target image is enhanced using the multi-scale Retinex algorithm. Convert the enhanced target image to the LAB color space; Illumination compensation is performed on the luminance channel of the target image, while the original chroma information of the chroma channel of the target image is preserved; Calculate the luminance variance corresponding to the luminance channel of the target image, and compare the luminance variance with a set threshold. In response to the brightness variance being greater than a set threshold, the scale parameter of the multi-scale Retinex algorithm or the illumination compensation intensity of the brightness channel is adjusted, and the target image is reprocessed.

3. The image multimodal feature extraction method according to claim 1, characterized in that, Extracting the texture and color features of the target image, including: An edge detection algorithm is used to identify the contour edges of ground features in the target image; The target image is partitioned based on the contour edges of the ground features, and the filtering window size of different partitions is limited. The median filtering algorithm is then used to filter and denoise the target image.

4. The image multimodal feature extraction method according to claim 1, characterized in that, Dynamically adjusting the weight ratios of the color and texture features includes: The weighted Euclidean distance between the target image and previously acquired historical images is calculated, and the change intensity index corresponding to the target image is calculated using an exponential smoothing algorithm. The average color saliency index of all historical images is calculated, and the weight ratio of the color feature and texture feature is adjusted by combining the color saliency index and change intensity index corresponding to the target image.

5. The image multimodal feature extraction method according to claim 1, characterized in that, Texture features are extracted from the target image using rotation-invariant coding and gray-level co-occurrence matrix, including: Construct an LBP coding matrix and a gray-level co-occurrence matrix based on the target image; The histogram feature vector of the target image is extracted using the LBP coding matrix, and the multidimensional directional feature vector of the target image is extracted using the gray-level co-occurrence matrix. The directional feature vector is fused with the histogram feature vector to generate the original feature pool, and the original feature pool is subjected to dimensionality reduction and redundant dimension filtering to obtain the texture features.

6. The image multimodal feature extraction method according to claim 1, characterized in that, The color features of the target image are analyzed collaboratively using LAB and HSV color spaces, including: The target image is converted into a LAB image and an HSV image respectively, and the LAB image and HSV image are aligned. The A-channel and B-channel images in the LAB image are divided into dense grids, and the main color clusters are identified by clustering based on the statistics of each grid cell to determine the structured chromaticity vector of the target image. Based on the saturation channel, the HSV image is divided into high-saturation and low-saturation regions. The hue histogram of the high-saturation region is calculated, and the low-saturation region is compensated by interpolation to determine the hue saliency vector of the target image. The HSV saturation weight is determined by the hue saliency vector, the chromaticity noise of the structured chromaticity vector is filtered by the HSV saturation weight, and the dynamic fusion coefficient is calculated based on the consistency of the main color distribution. Based on the dynamic fusion coefficient, the structured chromaticity vector and the hue salience vector are fused to generate a joint feature vector; The target image is divided into multiple sub-block images of the same size, and the joint feature vector is reduced in dimensionality. The color entropy and diversity index of each sub-block image are calculated to generate the color features of the target image.

7. A method for classifying and identifying land features, characterized in that, include: The image multimodal feature extraction method as described in any one of claims 1-6 is used to extract multimodal feature vectors from the target image; Based on the multimodal feature vectors corresponding to the target image, the ground features in the target image are classified using a trained random forest model, and corresponding ground feature boundaries are generated.

8. The method for classifying and identifying land features according to claim 7, characterized in that, Generate the corresponding ground feature boundaries, including: The random forest model is used to obtain the classification probability map corresponding to the target image, and multi-scale edge detection is performed based on the classification probability map to obtain the boundaries of different land features.

9. A GIS image generation method, characterized in that, include: The land feature classification and recognition method as described in claim 7 is used to classify and recognize the target image of the target area, and obtain the geographic boundary vector data corresponding to the target image. The geographic boundary vector data corresponding to the target image is mapped to a geographic coordinate system to generate a GIS image corresponding to the target area.

10. The GIS image generation method according to claim 9, characterized in that, Mapping the geographic boundary vector data to a geographic coordinate system includes: At least four uniformly distributed ground control points are selected from the geographic boundary vector data, and a six-parameter transformation matrix is ​​fitted based on all the selected ground control points. The pixel coordinates in the geographic boundary vector data are converted into planar projection coordinates using the six-parameter transformation matrix, and the planar projection coordinates are converted into geographic coordinates using ellipsoidal projection.

Images