A remote sensing image processing system and a processing method

Abstract

The present invention relates to the technical field of image extraction, and provides a remote sensing image processing system, including a server, as well as a drone, a data acquisition module, a feature extraction module, an image fusion module, an evaluation module, and a result output module. The data acquisition module dynamically acquires satellite images and image data obtained by the drone in the monitored area. The image fusion module preprocesses the satellite images and the image data to generate multi-spectral images and high-resolution images respectively, and performs fusion processing on the multi-spectral images and the high-resolution images to form a fused image. The feature extraction module extracts spectral features, texture features, and shape features from the fused image to generate a multi-dimensional feature description matrix. The evaluation module evaluates the abnormal positions based on the multi-dimensional feature description data to form an evaluation result. The result output module generates a warning prompt according to the evaluation result.

Description

Technical Field

[0001] The present invention relates to the technical field of image extraction, and particularly to a remote sensing image processing system and a processing method. Background Art

[0002] Remote sensing refers to an image data acquisition technology that detects and records the radiation and reflection characteristics of the electromagnetic waves of a target through a hyperspectral camera and an area array camera without contacting the target.

[0003] For example, a remote sensing image processing method and device disclosed in Chinese Patent CN101576994B selects sub-nodes with strong computing power to participate in the processing of remote sensing images according to the maximum data processing capacity of the sub-nodes in the grid, thereby improving the processing speed of remote sensing images; by dividing the remote sensing image into multiple remote sensing sub-images, the sub-nodes in the grid can perform parallel processing of the remote sensing sub-images according to the processing capabilities of the sub-nodes. Although the processing speed is improved, the consideration of the multi-source data fusion ability is ignored, resulting in a significant reduction in the reliability of image analysis.

[0004] In order to solve the problems commonly existing in this field, such as low processing efficiency, large loss of spectral information and spatial details, poor multi-source data fusion ability, low intelligence level, poor evaluation means, poor accuracy of classification and change detection, and insufficient visibility of analysis results, the present invention is made. Summary of the Invention

[0005] The purpose of the present invention is to propose a remote sensing image processing system and a processing method for the current existing deficiencies.

[0006] In order to overcome the deficiencies of the prior art, the present invention adopts the following technical solutions:

[0007] A remote sensing image processing system, the remote sensing image processing system includes a server and a drone, and the remote sensing image processing system further includes a data acquisition module, a feature extraction module, an image fusion module, an evaluation module, and a result output module. The server is respectively connected to the data acquisition module, the feature extraction module, the image fusion module, the evaluation module, and the result output module.

[0008] The data acquisition module dynamically acquires satellite images of the monitoring area and image data acquired by the drone. The image fusion module preprocesses the satellite images and the image data to generate multi-spectral images and high-resolution images respectively, and performs fusion processing on the multi-spectral images and the high-resolution images to form a fusion image. The feature extraction module extracts spectral features, texture features, and shape features from the fusion image to generate a multi-dimensional feature description matrix. The evaluation module evaluates the abnormal positions according to the multi-dimensional feature description data to form an evaluation result. The result output module generates a warning prompt according to the evaluation result.

[0009] The image fusion module includes a preprocessing unit, a grid division unit, an analysis unit, a weighted fusion unit, and a fusion optimization unit. The preprocessing unit preprocesses satellite images and image data to generate multispectral images and high-resolution images respectively. The grid division unit divides the input multispectral images and high-resolution images into grid cells of a fixed size. The analysis unit calculates the spectral consistency weights of the multispectral images within each grid cell and evaluates the richness of spatial details of the high-resolution images within each grid cell. The weighted fusion unit performs a fusion process based on the spectral consistency weights and detail richness weights. The fusion optimization unit performs a quality assessment and optimization on the preliminary fused image to form a fused image.

[0010] Among them, the fusion optimization unit evaluates the information content and quantization evaluation index of the preliminary fused image. If the index does not meet the monitoring threshold of the system, the size and weights of the grid cells are readjusted.

[0011] Optionally, the data acquisition module includes an image acquisition unit and an image collection unit. The image acquisition unit retrieves satellite image data of the monitoring area, and the image collection unit collects image data acquired by a drone. The image collection unit is mounted on the drone and follows the drone to collect image data of the monitoring area.

[0012] Optionally, the grid division unit includes a spatial alignment component, a grid generation component, a multi-source mapping grid mapping component, and a grid quality control component. The spatial alignment component aligns the spatial coordinate systems of the multispectral image and the high-resolution image. The grid generation component generates grid cells of a fixed size according to the spatial range and resolution of the image. The multi-source mapping grid mapping component maps the pixel values of the multispectral image and the high-resolution image to the corresponding grid cells. The grid quality control component evaluates the quality assessment result of the grid division and adjusts the grid size or range according to the evaluation result.

[0013] Optionally, the analysis unit includes a spectral consistency component, a spatial detail evaluation component, and a weight normalization component. The spectral consistency component analyzes the spectral distribution characteristics of the multispectral image within each grid cell and calculates the spectral consistency weights. The spatial detail evaluation component evaluates the richness of details of the high-resolution image within each grid cell. The weight normalization component normalizes the spectral consistency weights and detail richness weights.

[0014] Optionally, the weighted fusion unit includes a weight mapper, a pixel fuser, a boundary smoothing component, and a global adjustment component. The weight mapper receives the spectral consistency weight and the detail richness weight calculated by the analysis unit and maps them to the pixel level of each grid cell. The pixel fuser performs weighted superposition on the pixels within each grid cell to generate a preliminary fused image.

[0015] The boundary smoothing component smooths the boundaries between grid cells, and the global adjustment component performs global adjustment on the fused preliminary image.

[0016] Optionally, the feature extraction module includes a spectral feature extraction unit, a texture feature extraction unit, a shape feature extraction unit, and a multi-dimensional feature encoding unit. The spectral feature extraction unit analyzes the spectral information of the fused image. The texture feature extraction unit analyzes the texture characteristics of the ground objects using the image gray-scale change information. The shape feature extraction unit analyzes the shape features of the ground objects in the fused image based on the edges and geometric structures. The multi-dimensional feature encoding unit integrates the extracted spectral, texture, and shape features into a unified multi-dimensional feature description matrix F(x, y) and normalizes it to form a normalized multi-dimensional feature description matrix F’(x, y).

[0017] Optionally, the evaluation module calculates the comprehensive anomaly index Abnormal(x, y) of the pixel point (x, y) based on the multi-dimensional feature description matrix F’(x, y) according to the following formula:

[0018]

[0019] where d is the number of feature dimensions of the normalized multi-dimensional feature description matrix F’(x, y), Z i (x, y) is the normalized anomaly score of the pixel point (x, y) in the i-th feature dimension, CV i is the coefficient of variation of the i-th feature dimension, and α is an adjustment coefficient, whose value is set by the system according to the actual situation;

[0020] If the comprehensive anomaly index Abnormal(x, y) of the pixel point (x, y) does not fall within the evaluation threshold range [B1, B2] set by the system, it indicates that the pixel point (x, y) is abnormal.

[0021] In addition, the present invention also provides a remote sensing image processing method, and the remote sensing image processing method includes the following steps:

[0022] S1. Dynamically obtain the satellite images of the monitoring area and the image data collected by the UAV through the data acquisition module;

[0023] S2. Preprocess the satellite image and image data through the image fusion module to generate a multispectral image and a high-resolution image respectively, and perform fusion processing on the multispectral image and the high-resolution image to form a fused image;

[0024] S3. Extract spectral features, texture features, and shape features from the fused image through the feature extraction module to generate a multi-dimensional feature description matrix;

[0025] S4. Evaluate the abnormal position according to the multi-dimensional feature description data through the evaluation module to form an evaluation result;

[0026] S5. Generate a warning prompt according to the evaluation result through the result output module.

[0027] The beneficial effects achieved by the present invention are as follows: Through the mutual cooperation of the data acquisition module, the feature extraction module, the image fusion module, the evaluation module, and the result output module, the remote sensing image processing system has high efficiency, accuracy, and operability in multi-source data fusion, anomaly detection, and result output, ensuring that the entire system can analyze and warn the monitored area efficiently and comprehensively in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the drawings are not necessarily drawn to scale, but the emphasis is placed on showing the principles of the embodiments. In different views, the same reference numerals designate the same parts.

[0029] Figure 1 It is a schematic overall block diagram of the present invention.

[0030] Figure 2 It is a schematic block diagram of the image fusion module of the present invention.

[0031] Figure 3 It is a schematic extraction block diagram of the feature extraction module of the present invention.

[0032] Figure 4 It is a schematic flowchart of the remote sensing image processing method of the present invention.

[0033] Figure 5 It is a schematic diagram of the report generated by the report generation unit of the present invention for a certain area.

[0034] Figure 6 It is a schematic diagram of the effects of the prior art and the technology of the present application of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] The following embodiments will further elaborate on the related technical content of the present invention, but the disclosed content is not intended to limit the protection scope of the present invention.

[0036] Example 1: According toFigure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 As shown in , this embodiment provides a remote sensing image processing system. The remote sensing image processing system includes a server and an unmanned aerial vehicle (UAV). The remote sensing image processing system further includes a data acquisition module, a feature extraction module, an image fusion module, an evaluation module, and a result output module. The server is respectively connected to the data acquisition module, the feature extraction module, the image fusion module, the evaluation module, and the result output module, and stores the intermediate data and process data of the data acquisition module, the feature extraction module, the image fusion module, the evaluation module, and the result output module in the database of the server for querying and calling. The data acquisition module dynamically acquires satellite images of the monitored area and image data acquired by the UAV. The image fusion module preprocesses the satellite images and the image data to generate multi-spectral images and high-resolution images respectively, and performs fusion processing on the multi-spectral images and the high-resolution images to form a fused image. The feature extraction module extracts spectral features, texture features, and shape features from the fused image to generate a multi-dimensional feature description matrix. The evaluation module evaluates the abnormal positions based on the multi-dimensional feature description data to form an evaluation result. The result output module generates a warning prompt according to the evaluation result. In this embodiment, the remote sensing image processing system further includes a central processing unit (CPU). The CPU is respectively connected to the data acquisition module, the feature extraction module, the image fusion module, the evaluation module, and the result output module for control connection, and centrally controls the data acquisition module, the feature extraction module, the image fusion module, the evaluation module, and the result output module based on the CPU, and stores the control data of the CPU in the server to ensure the reliability and accuracy of the entire system for remote sensing image processing.

[0037] The image fusion module includes a preprocessing unit, a grid division unit, an analysis unit, a weighted fusion unit, and a fusion optimization unit. The preprocessing unit preprocesses the satellite images and the image data to generate multi-spectral images and high-resolution images respectively. The grid division unit divides the input multi-spectral images and high-resolution images into grid units of a fixed size. The analysis unit calculates the spectral consistency weights of the multi-spectral images in each grid unit and evaluates the spatial detail richness of the high-resolution images in each grid unit. The weighted fusion unit performs fusion processing based on the spectral consistency weights and the detail richness weights. The fusion optimization unit performs quality evaluation and optimization on the preliminary fused image to form a fused image. Among them, the fusion optimization unit evaluates the information content and quantization evaluation index of the preliminary fused image. If the index does not meet the monitoring threshold of the system, the size and weights of the grid units are readjusted.

[0038] Optionally, the data acquisition module includes an image acquisition unit and a picture acquisition unit. The image acquisition unit retrieves satellite image data of the monitoring area, and the picture acquisition unit acquires picture data obtained by the UAV. The picture acquisition unit is mounted on the UAV and follows the UAV to acquire picture data of the monitoring area. The image acquisition unit includes a call instructing device and a data receiver. The call instructing device sends a call instruction to the satellite, and the call instruction includes the geographical boundary, time range, and resolution requirement of the monitoring area. The data receiver receives the satellite image data fed back by the satellite. In this embodiment, the satellite image data fed back by the satellite includes the following content: data timestamp, satellite name (such as Sentinel-2), spatial resolution and spectral range of the available image, and image download link. The picture acquisition unit includes a data memory and a picture acquisition probe. The picture acquisition probe acquires picture data of the monitoring area, and the data memory stores the picture data of the monitoring area acquired by the picture acquisition probe. During the process of acquiring the picture data of the monitoring area, the UAV is controlled to conduct patrol inspection of the set monitoring area to obtain the picture data of the monitoring area. At the same time, the monitoring area patrolled by the UAV should be consistent with the geographical boundary or monitoring location called by the satellite.

[0039] In this embodiment, how to control the UAV to conduct patrol inspection in the monitoring area is a conventional technical means, so it will not be elaborated one by one in this embodiment.

[0040] The steps for the preprocessing unit to preprocess the satellite image and the picture data acquired by the UAV include:

[0041] S100. Input the satellite image and the UAV image data:

[0042] S101. Conduct geometric correction, radiometric correction, and cropping processing on the satellite image and the UAV picture data respectively;

[0043] Among them, conducting geometric correction, radiometric correction, and cropping processing on the satellite image and the UAV picture data is an image processing technology and a technical means well-known to those skilled in the art.

[0044] Optionally, the grid division unit includes a spatial alignment component, a grid generation component, a multi-source mapping grid mapping component, and a grid quality control component. The spatial alignment component aligns the spatial coordinate systems of the multi-spectral image and the high-resolution image. The grid generation component generates grid cells of a fixed size based on the spatial range and resolution of the images. The multi-source mapping grid mapping component maps the pixel values of the multi-spectral image and the high-resolution image to the corresponding grid cells. The grid quality control component evaluates the quality assessment results of the grid division and adjusts the grid size or range according to the evaluation results. Evaluate the coverage of the grid division. If there is insufficient valid data in some grid cells, readjust the grid size or range. The spatial alignment component aligns the spatial coordinate systems of the multi-spectral image and the high-resolution image to ensure that the two images have the same geographic reference system (CRS, Coordinate Reference System), facilitating subsequent grid division and data mapping

[0045] For the geographic boundaries of the multi-spectral image and the high-resolution image, they are respectively represented as: Multi-spectral image boundary: [x min 多 , x max 多 , y min 多 , y max 多 . High-resolution image boundary: [x min 高 , x max 高 , y min 高 , y max 高 .

[0046] Overlapping part (intersection): x min = max(x min 多 , x min 高 ), x max = min(x max 多 , x max 高 ); y min = max(y min 多 , y min 高 ), y max = min(y max 多 , y max 高 );

[0047] Fully covered part (union): xmin = min(x min 多 , x min 高 ), x max = max(x max 多 , x max 高 );

[0048] y min = min(y min 多 , y min 高 ), y max = max(y max 多 , y max 高 );

[0049] The grid generation component determines the grid cell size: Set the size s of the grid cell: s = R t * k; where R t is the resolution of the image, k is the scaling factor (default is 1), and the value range is [0.1, 10], or set according to the scenario.

[0050] In this embodiment, an example of the value of the scaling factor k is provided: In the scenario of large-scale land use classification, the scaling factor k = 10; in the scenario of vegetation monitoring in precision agriculture, the scaling factor k = 1; in the scenario of urban heat island effect analysis, the scaling factor k = 4; in the scenario of refined monitoring of natural disasters, the scaling factor k = 0.5; In short, for different regional analyses, different scaling factors are selected. For example: for large-area analysis: k > 1, the grid cells become larger; for refined analysis: k < 1, the grid cells become smaller. The grid generation component generates a grid network according to the following formula:

[0051]

[0052] where each grid cell is numbered G ij , i, j are row and column indices. is the floor symbol. The multi-source mapping grid mapping component assigns pixels to grid cells: The grid generation component calculates the grid cell G ij to which each pixel point (x, y) belongs:

[0053]

[0054] where is the ceiling symbol, and i, j are row and column indices; The grid generation component for each grid cell G ij, calculate the statistical features of the included multi-spectral pixel values and high-resolution pixel values:

[0055]

[0056] In the formula, F ij 多光谱 is the average pixel value of the multi-spectral image in the grid cell G ij , F ij 高分辨率 is the average pixel value of the multi-spectral image in the grid cell G ij , N and M are the numbers of multi-spectral pixels and high-resolution pixels in the grid respectively, I k 多光谱 is the value of the k-th multi-spectral pixel in the grid cell G ij , I k 高分辨率 is the value of the k-th high-resolution pixel in the grid cell G ij . When the eigenvalue F ij 多光谱 and F ij 高分辨率 of each grid cell are calculated, they are stored.

[0057] During the process of the grid quality control component evaluating the quality of grid division, calculate each grid cell G ij , calculate the proportion C ij of valid data pixels:

[0058] Among them, N 有效 is the number of valid data pixels in the grid cell, N 总 is the total number of pixels in the grid cell, and its value is calculated according to the following formula:

[0059] In the formula, s is the side length of the grid cell (unit: meter), R is the resolution of the image (unit: meter / pixel), R 2 is the area of a single pixel.

[0060] The determination of valid pixels is achieved by traversing all pixel values I k in the grid cell, and counting the number of pixels that meet the valid conditions:

[0061] In the formula, Ⅱ(·) is the indicator function: Among them, I k is the pixel value of the k-th pixel in the grid cell.

[0062] Among them, an invalid value refers to a pixel value that cannot be used for effective calculation and analysis. In this embodiment, abnormal invalid values are identified through a statistical method, that is, the mean μ' and standard deviation σ' of the pixel values are calculated, and pixels deviating from the mean by more than a threshold (such as 3σ') are regarded as invalid values;

[0063] The process of determining and adjusting the validity of raster cells includes: The first step: Determine whether the raster cell is valid: If the proportion C of valid data pixels ij is less than the coverage threshold T set by the system 覆盖度 , then the raster cell is considered invalid; that is, if there is insufficient valid data in a certain raster (such as fewer valid pixels or too many invalid values), the raster cell will be marked as invalid. The second step: Statistically calculate the proportion of invalid rasters; if the proportion of invalid rasters exceeds the set threshold R 无效 , adjust the size of the raster cell, where the new raster size s 新 is calculated according to the following formula: s 新 = s·f, f ∈ [0.5, 2]; in the formula, s is the original raster size;

[0064] Among them, f is an adjustment factor, and the raster size is increased according to insufficient coverage or decreased according to over-dense coverage. After adjusting the raster cell size, a new raster is generated and mapped.

[0065] The analysis unit includes a spectral consistency component, a spatial detail evaluation component, and a weight normalization component. The spectral consistency component analyzes the spectral distribution characteristics of the multi-spectral image in each raster cell and calculates the spectral consistency weight. The spatial detail evaluation component evaluates the richness of details of the high-resolution image in each raster cell, and the weight normalization component normalizes the spectral consistency weight and the detail richness weight.

[0066] The process by which the spectral consistency component analyzes the spectral distribution characteristics of the multi-spectral image in each raster cell and calculates the spectral consistency weight includes: S300. Extract the multi-spectral data within the raster cell.

[0067] S301. Calculate the spectral mean μ and standard deviation σ:

[0068] In the formula, X i represents the average value of all pixel values of the raster cell in the i-th band of the multi-spectral image, that is, the mean value of the pixel values in the i-th band within the raster cell, and n is the number of bands.

[0069] S303. Calculate the coefficient of variation CV as the spectral consistency index:

[0070] S304. Calculate the spectral consistency weight ω 光谱,ij : ω 光谱,ij= 1 - CV; In this example, the smaller the coefficient of variation CV, the higher the spectral consistency and the greater the weight

[0071] The evaluation process of the spatial detail evaluation component includes: 1) Extracting high-resolution image data within the grid cell. 2) Using an edge detection algorithm (such as the Sobel or Canny operator) to extract image edges.

[0072] In this example, the edge detection formula (taking Sobel as an example):

[0073]

[0074] In the formula, I(x, y) is the grayscale value matrix within the grid cell, representing the grayscale intensity of each pixel within the grid range, G x and G y respectively represent the gradients of the image in the x and y directions, and G is the edge intensity. Among them, edge detection (such as the Sobel operator) is an algorithm that extracts edge features based on the change of image gradients. The grayscale value can directly reflect the intensity change between pixels. Therefore, it is necessary to convert the high-resolution image data into a grayscale image for processing. For a color image (such as an RGB image), it needs to be converted into a grayscale image first:

[0075] I(x, y) = 0.2989·R(x, y) + 0.5870·G(x, y) + 0.1140·B(x, y); In the formula, R(x, y), G(x, y), and B(x, y) are the red, green, and blue channel values of the pixel point respectively.

[0076] 3) Calculate the edge density D 边缘 :

[0077] In the formula, the number of edge pixels is the number of pixels identified as edges in the grid cell, N 总 is the total number of pixels in the grid cell. Among them, the number of edge pixels calculates the gradient intensity of the pixels within the grid cell through the edge detection algorithm. Specifically: using the Sobel operator, and when G k > T 边缘 , then mark this pixel as an edge pixel. Among them, G k is the edge intensity of the kth pixel within the grid cell, and its value is calculated by the Sobel operator;

[0078] In addition, T 边缘 is the threshold of the edge intensity, used to distinguish edge pixels from non-edge pixels, and its value is set by the system, that is, a fixed value is preset. For example, according to experimental experience, set T 边缘 = 50 (the intensity range is usually 0 - 255). Count the number of edge pixels: Traverse all the pixels within the grid cell and count the number of pixels that meet the conditions:

[0079] In the formula, Ⅱ(·) is the indicator function:

[0080] 4) Use D 边缘 to represent the detail richness weight ω 细节,ij ; the weight normalization component normalizes the spectral consistency weight ω 光谱,ij and the detail richness weight ω 细节,ij for all grid cells:

[0081]

[0082] In the formula, ω′ 光谱,ij is the normalized spectral weight, ω′ 细节,ij is the normalized detail weight, w 光谱,ij is the spectral consistency weight, w 细节,ij is the detail richness weight, w 光谱，min is the minimum value of the spectral consistency weight, w 光谱，max is the maximum value of the spectral consistency weight, w 细节，min is the minimum value of the spectral consistency weight, w 细节，max is the maximum value of the spectral consistency weight; output the normalized weights ω′ 光谱,ij and ω′ 细节,ij of each grid cell to the weighted fusion unit.

[0083] Optionally, the weighted fusion unit includes a weight mapper, a pixel fuser, a boundary smoothing component, and a global adjustment component. The weight mapper receives the spectral consistency weight and the detail richness weight calculated by the analysis unit and maps them to the pixel level of each grid cell. The pixel fuser performs weighted superposition on the pixels within each grid cell to generate a preliminary fused image. The boundary smoothing component smooths the boundaries between grid cells to eliminate the image discontinuity problem caused by grid division. The global adjustment component performs global adjustment on the preliminary fused image to ensure the balance of overall spectral information and detail information.

[0084] The weight mapper maps the spectral consistency weight and the detail richness weight to the pixel level of each grid cell according to the following steps: S400. Receive the spectral consistency weight and the detail richness weight calculated by the analysis unit. During the assignment process, traverse all grid cells and store the normalized weights ω′ 光谱,ij and ω′ 细节,ij ;

[0085] S401. Determine the pixel range of the grid cell: According to the size s of the grid division and the image resolution R, determine the number of pixels N within each grid cell G ij :

[0086] Assume that the boundary range of the grid cell G ij is: row range = [i start , i end , column range = [j start , j end ;

[0087] S402. Assign a unified weight value: For each grid cell G ij , assign its normalized spectral weight ω′ 光谱,ij and normalized detail weight ω′ 细节,ij to all pixels within this grid cell:

[0088] ω′ 光谱,k = ω′ 光谱,ij , ω′ 细节,k = ω′ 细节,ij , k ∈ G ij ;

[0089] In the formula, ω′ 光谱,k is the pixel-level spectral weight, and ω′ 细节,k is the pixel-level detail weight;

[0090] In the above formula, the meaning of the whole formula is: Assign the spectral weight and detail weight at the grid level to all pixels within this grid; specifically, each grid cell G ij is determined by the grid division size s and the image resolution R; the grid division ensures that each ij corresponds to a unique spatial area. i start and i end represent the starting and ending points of the pixel rows of this grid; j start and j end represent the starting and ending points of the pixel columns of this grid; all pixels k of each grid cell G ij are determined. Therefore, the grid-level weights ω′ 光谱,ij and ω′ 细节,ij are directly assigned to all pixels k of G ij . In other words, each pixel k can be uniquely mapped to a certain grid cell G ij according to its row and column coordinates.

[0091] S403. Store the pixel weights after mapping: Record the spectral and detail weight values ω′ 光谱,k and ω′ 细节,k for each pixel k.

[0092] S404. Generate pixel-level weight output: Store the weights of each pixel in the image data format for facilitating fusion with multi-spectral and high-resolution images.

[0093] Create two weight maps with the same resolution as the original image: a spectral weight map \(W\) spec : The value of each pixel is \(\omega'\) 光谱,k ; a detail weight map \(W\) detail : The value of each pixel is \(\omega'\) 细节,k ;

[0094] Among them, the spectral weight map \(W\) spec is a matrix consistent with the resolution of the input image, where the value of each pixel is equal to the normalized spectral consistency weight \(\omega'\) of the grid cell where the pixel is located 光谱,k :

[0095] \(W\) spec (x,y)=\(\omega'\) 光谱，k , \(k\in G\) ij ;

[0096] In the formula, \((x,y)\) represents the position of the pixel in the full-image coordinates, \(k\) is the index of the pixel, and \(G\) ij is the grid cell where the pixel is located.

[0097] The detail weight map \(W\) detail is a matrix consistent with the resolution of the input image, where the value of each pixel is equal to the normalized detail richness weight \(\omega'\) of the grid cell where the pixel is located 细节,k : \(W\) detail (x,y)=\(\omega'\) 细节,k , \(k\in G\) ij ;

[0098] In the formula, \((x,y)\) represents the position of the pixel in the full-image coordinates, \(k\) is the index of the pixel, and \(G\) ij is the grid cell where the pixel is located.

[0099] The generation process of the weight map includes the following steps:

[0100] S1000. Input data: grid-level weight matrix: \(\omega'\) 光谱,ij is the normalized spectral weight for each grid cell; \(\omega'\) 细节,ij is the normalized detail weight for each grid cell. Grid division size: side length \(s\) of the grid cell; resolution \(R\) of the input image.

[0101] S1001. Map to pixel level: Determine the pixel range corresponding to the grid cell: Each grid cell \(G\) ij contains the pixel row and column ranges:

[0102] For example: If the grid size \(s = 100m\) and the image resolution \(R = 10m\), then the pixel range corresponding to each grid is \(10\times10\). Assign a unified weight value: Assign the grid-level weights \(\omega'\) 光谱,ij and \(\omega'\)细节,ij All pixels allocated to the grid cell.

[0103] S1002. Store the weight maps: spectral weight map W spec : At each pixel position (x, y), fill in the corresponding spectral weight value. Detail weight map W detail : At each pixel position (x, y), fill in the corresponding detail weight value.

[0104] The pixel fuser performs a weighted calculation on the pixel values of the multispectral image and the high-resolution image according to the spectral consistency weight and the detail richness weight. Specifically, read the included multispectral pixel I k 多光谱 and the high-resolution pixel I k 高分辨率 , as well as the spectral weight map W spec (x, y), detail weight map W detail (x, y);

[0105] For each pixel (x, y), fuse according to the following formula:

[0106]

[0107] Fuse the fused I 融合 (x, y) of each pixel (x, y) according to the above formula, and form a preliminary fused image I 融合 (x, y) with the fused pixel values. 初步融合 .

[0108] The boundary smoothing component smooths the boundary region between grid cells, eliminates the image discontinuity problem caused by grid division, and makes the image transition more natural. The specific steps include: 1) Detect the boundary region: Mark the boundary region of each grid cell, that is, through the row and column ranges of the grid index G ij : row range = [i start , i end ; column range = [j start , j end ; Mark the pixels in the boundary region as the transition region. 2) Smoothing process: Use a Gaussian blur convolution kernel to smooth the boundary region:

[0109]

[0110] where: K(m, n) is the Gaussian kernel: m and n are the row and column distances of pixel offset;

[0111] The size of the kernel is usually determined by the range of 6σ (i.e., the boundary of the kernel is 3 times σ away from the center), where σ is a parameter of the blurring degree, and its value range is: (0, ∞). When σ → 0, it means no blurring and the image remains completely clear. When σ → ∞, the image is extremely blurred and tends to a single color. In this embodiment, σ usually takes values in: [0.5, 10]; lower σ values (such as 0.5 - 2) are used for fine blurring or noise suppression; higher σ values (such as 5 - 10) are used for smoothing of larger areas or boundary transitions.

[0112] The steps of the global adjustment component performing global adjustment on the preliminary image after fusion to ensure the overall balance of spectral information and detail information include:

[0113] S500, Spectral equalization: Perform spectral adjustment on the smoothed image to make the distribution of spectral information consistent with that of the multi - spectral image: I 光谱 (x, y) = I 平滑 (x, y)+λ·(I 多光谱 (x, y)-I 平滑 (x, y));

[0114] In the formula, λ is the spectral adjustment coefficient, and its value is set by the system, with the value range of [0, 1], which is used to balance spectral and detail information. I 光谱 (x, y) is the pixel value after spectral equalization, I 平滑 (x, y) is the preliminary fusion image I 初步融合 (x, y) is the image pixel value after being processed by the boundary smoothing component of the preliminary fusion image I 多光谱 (x, y) is the original pixel value of the multi - spectral image at the pixel (x, y); among them, when λ = 0, the value of the smoothed image I 平滑 is completely adopted without considering the characteristics of the multi - spectral image; when λ = 1, it is completely adjusted according to the spectral characteristics of the multi - spectral image, ignoring the influence of the smoothed image;

[0115] S501, Detail enhancement: Extract the detail information (such as edges and textures) of the high - resolution image and superimpose it on the spectrally adjusted image: I 增强 (x, y) = I 光谱 (x, y)+β·ΔI 高分辨率 (x, y);

[0116] In the formula, I 光谱 (x, y) is the image pixel value after spectral equalization, β is the detail enhancement coefficient, which controls the influence weight of the high - resolution details in the enhanced image, and ΔI 高分辨率 (x, y) is the high - frequency component of the high - resolution image;

[0117] For ΔI 高分辨率 (x, y) being the high - frequency component of the high - resolution image is calculated according to the following formula:

[0118] ΔI 高分辨率 (x,y) = I 高分辨率 (x,y) - Gaussian blur(I 高分辨率 (x,y));

[0119] In the formula, I 高分辨率 (x,y) is the original pixel value of the high - resolution image at pixel (x,y), and Gaussian blur(I 高分辨率 (x,y)) is the low - frequency part obtained by performing Gaussian blur processing on the high - resolution image I 高分辨率 (x,y). Its value is obtained by using the Gaussian blur algorithm to process the image, removing high - frequency details and retaining the low - frequency components of the image. The low - frequency components correspond to the regions in the image where the brightness of the object is uniform or changes slowly. Those skilled in the art are familiar with this, so it will not be elaborated in this embodiment.

[0120] S502, Global equalization: Perform global histogram equalization on the adjusted image I 增强 to improve the dynamic range and visual effect of the image.

[0121] S503, Generate the final fused image: Output the adjusted image I 最终融合 , which contains balanced spectral and detail information.

[0122] The fusion optimization unit evaluates the quality of the preliminary fused image and adjusts and optimizes the image according to the set evaluation metrics, thereby generating a high - quality final fused image. Among them, the fusion optimization unit quantifies the evaluation metrics (including spectral information fidelity Q 光谱 and detail information retention Q 细节 ) to determine whether the spectral information fidelity, detail information retention, and overall quality of the preliminary fused image meet the system requirements. Specifically, the spectral information fidelity Q 光谱 is calculated according to the following formula:

[0123] In the formula, I 多光谱 (x,y) is the pixel value of the multi - spectral original image, and its value can be directly obtained from the image data through the pixel value of the multi - spectral image. I 融合 (x,y) is the pixel value of the preliminary fused image. is the global mean value of the multi - spectral image. Among them, the global mean value of the multi - spectral image is calculated according to the following formula:

[0124]

[0125] In the formula, M is the image width (number of columns), and N is the image height (number of rows).

[0126] Detail information retention Q细节 Calculate according to the following formula:

[0127]

[0128] In the formula, is the gradient vector of the fused image, is the gradient vector of the high-resolution image; among them, for the gradient of the fused image Calculate according to the following formula:

[0129]

[0130] In the formula, G x 融合 is the gradient of the fused image in the horizontal direction (column direction), G y 融合 is the gradient of the fused image in the vertical direction (row direction), is the gradient magnitude of the fused image;

[0131] For the gradient of the high-resolution image Calculate according to the following formula:

[0132]

[0133] In the formula, G x 高分辨率 is the gradient of the high-resolution image in the horizontal direction (column direction), G y 高分辨率 is the gradient of the high-resolution image in the vertical direction (row direction), is the magnitude of the gradient of the high-resolution image;

[0134] The overall quality index Q 总 Calculate according to the following formula: Q 总 = η·Q 光谱 + λ·Q 细节 ;

[0135] In the formula, η and λ are weight coefficients, usually satisfying η + λ = 1. Among them, when the application scenario pays more attention to spectral fidelity (such as crop classification, vegetation monitoring, etc.), η should be larger; when the application scenario pays more attention to spatial details (such as building boundary detection, road monitoring, etc.), λ should be larger. When it is necessary to balance spectral and detail information (such as disaster monitoring, mixed land cover change analysis), then η ≈ λ.

[0136] In addition, in this embodiment, an example of the value of the weight coefficient is also provided. Specifically: 1) In the scenarios of agriculture and vegetation monitoring (paying more attention to spectral information to distinguish different crop or vegetation types), then η = 0.7, λ

[0137] = 0.3; 2) In the scenario of urban building and infrastructure monitoring (more concerned about spatial details to accurately detect building boundaries or road distributions), then η = 0.3 and λ = 0.7; 3) In the scenario of mixed scenarios (disaster monitoring, land use analysis) (requiring comprehensive spectral and detail information to ensure the comprehensiveness of change analysis), then η = 0.5 and λ = 0.5; 4) In the scenario of water body and pollution monitoring (concerned about changes in spectral characteristics (such as the change in reflectivity with the degree of pollution)), then η = 0.8 and λ = 0.2;

[0138] If Q 总 does not reach the set threshold, then enter the optimization process; otherwise, perform fusion processing with the current spectral consistency weight and detail richness weight;

[0139] The steps of optimization include: S600, weight redistribution: dynamically adjust the spectral weight W 光谱 and the detail weight W 细节 , where the new spectral weight W' 光谱 and the new detail weight W' 细节 are determined according to the following formula:

[0140]

[0141] In the formula, W 光谱 is the original spectral weight, W 细节 is the original detail weight, γ and δ are optimization factors that control the intensity of weight adjustment, and the value range is: γ, δ ∈ [0, 2]. In this embodiment, an example of the value of the optimization factor is provided. Specifically: 1) Vegetation monitoring (spectral information is prioritized). The quality of hyperspectral information is crucial for vegetation classification, and it is necessary to prioritize improving the spectral weight, then γ = 0.8 and δ = 0.3. 2) Building boundary monitoring (detail information is prioritized). Higher resolution and geometric details are required, and the detail weight is prioritized, then γ = 0.3 and δ = 0.9. 3) Disaster monitoring (combining spectral and detail information). It is necessary to balance spectral and detail information to analyze changes in mixed ground object areas, then γ = 0.5 and δ = 1.5. 4) Water body pollution monitoring (spectral information is extremely important). High spectral accuracy is crucial for detecting the degree of pollution and water body components, then γ = 1.0 and δ = 0.2.

[0142] S601, perform quality assessment on the optimized image again. If the target is not reached, repeat the above steps.

[0143] The fusion optimization unit determines whether to adjust the grid size by evaluating the following two key indicators (including the information amount indicator H 融合 and the edge density D 边缘 ) of the preliminary fusion image:

[0144] Wherein, L is the number of gray levels. For an 8-bit gray image, L = 256 (the gray value range is 0 to 255), and P i is the probability of the i-th gray value, and its value satisfies: n i is the number of pixels with the gray value of i, N is the total number of pixels in the image, and the calculation formula is: N = M × N, where M is the width of the image (number of pixels), and N is the height of the image (number of pixels).

[0145] The specific calculation process is as follows: 1) Calculate the gray histogram: count the gray values of all pixels in the image to generate a gray histogram. The number of pixels n i at each gray level i is the number of levels corresponding to the gray level in the histogram. 2) Calculate the gray probability P i ; 3) Calculate the information quantity index H 融合 .

[0146] If H 融合 or D 边缘 does not meet the threshold range set by the system (such as too low information quantity or insufficient edge information), trigger the grid size adjustment; specifically, if both H 融合 and D 边缘 do not meet the threshold range set by the system, that is, H 融合 does not fall into the information quantity monitoring range [TH2, TH2] set by the system, and D 边缘 does not fall into the edge density monitoring range [TD2, TD2] set by the system, then trigger a relatively large adjustment of the grid; if either H 融合 or D 边缘 does not meet the threshold range set by the system, that is, H 融合 does not fall into the information quantity monitoring range [TH2, TH2] set by the system, or D 边缘 does not fall into the edge density monitoring range [TD2, TD2] set by the system, then trigger a relatively small adjustment of the grid;

[0147] Among them, the information quantity monitoring range [TH2, TH2] set by the system and the edge density monitoring range

[0148] [TD2, TD2] are both set by the system according to the actual situation, which is a well-known technical means in the art, so it will not be elaborated one by one in this embodiment. In this embodiment, when both H 融合 and D 边缘 do not meet the threshold range set by the system, the steps of adjusting the grid size (steps S700 to S704) include:

[0149] S700. Calculate the new grid size, that is, take a relatively large reduction in the grid size to improve the resolution to capture more details and information quantity: The relatively large increase in the grid size is determined according to the following formula:

[0150] s 新 = s 当前 ·(1 + α);

[0151] In the above formula, s 当前 is the size of the current grid cell, s 新 is the adjusted size of the grid cell, and α is the adjustment ratio coefficient, usually in the range of [0.1, 0.5].

[0152] S701. Re - divide the grid: According to the new grid size, re - divide the multi - spectral image and the high - resolution image.

[0153] The boundary range of each grid cell is updated to: G ij = [i·s 新 , (i + 1)·s 新 ) × [j·s 新 , (j + 1)·s 新 );

[0154] S702. Recalculate the grid - level weights: According to the new grid division, recalculate the spectral consistency weight and the detail richness weight;

[0155] S703. Re - fuse and optimize: Based on the new grid division and weights, re - perform pixel - level weighted fusion and subsequent optimization.

[0156] S704. Dynamic adjustment and iteration: Continuously monitor the quality metrics H 融合 and D 边缘 .

[0157] If the adjusted grid size still does not meet the quality requirements, repeat the adjustment process until the system threshold is met or the maximum number of adjustments is reached.

[0158] Through the cooperation of the data acquisition module and the image fusion module, the efficient fusion of multi - source data (satellite images and UAV images) is achieved, making the output image have both the fidelity of spectral information and high - resolution spatial details, ensuring the information integrity and quality of multi - source remote sensing data after fusion.

[0159] If H 融合 or D 边缘 both meet the system - set threshold range, that is, H 融合 falls into the system - set information amount monitoring range [TH2, TH2], and D 边缘 falls into the system - set edge density monitoring range [TD2, TD2], then keep the grid size unchanged. If H 融合 or D 边缘 either one does not meet the system - set threshold range, that is, H融合 Does not fall within the information volume monitoring range [TH2, TH2] set by the system, but D 边缘 Falls within the edge density monitoring range [TD2, TD2] set by the system, or H 融合 Falls within the information volume monitoring range [TH2, TH2] set by the system, but D 边缘 When it does not fall within the edge density monitoring range [TD2, TD2] set by the system, steps S800 to S804 are executed:

[0160] S800. Calculate the new grid size, and then perform a small-scale grid adjustment;

[0161] If H 融合 Does not meet (insufficient information volume) (i.e., H 融合 Does not fall within the information volume monitoring range [TH2, TH2] set by the system): Slightly reduce the grid size to increase the information volume.

[0162] If D 边缘 Does not meet (insufficient details) (i.e., H 融合 Falls within the information volume monitoring range [TH2, TH2] set by the system): Slightly reduce the grid size to enhance the edge information.

[0163] Among them, the small-scale reduction of the grid size is determined according to the following formula: s 新 = s 当前 ·(1 - β); in the formula, s 当前 is the size of the current grid cell, s 新 is the size of the adjusted grid cell, and β is the adjustment ratio coefficient, usually in [0.1, 0.5].

[0164] S801. Re-divide the grid: Re-divide the multi-spectral image and the high-resolution image according to the new grid size. The boundary range of each grid cell is updated to:

[0165] G ij = [i·s 新 , (i + 1)·s 新 ) × [j·s 新 , (j + 1)·s 新 );

[0166] S802. Re-calculate the grid-level weights: Re-calculate the spectral consistency weight and the detail richness weight according to the new grid division;

[0167] S803. Re-fuse and optimize: Based on the new grid division and weights, re-perform pixel-level weighted fusion and subsequent optimization.

[0168] S804. Dynamically adjust and iterate: Continuously monitor the quality index H of the fused image融合 and D 边缘 。

[0169] If the adjusted grid size still does not meet the quality requirements, repeat the adjustment process until the system threshold is met or the maximum number of adjustments is reached.

[0170] In this embodiment, a provided adjusted proportionality coefficients α, β have a value range of: [0.1, 0.5], that is, an adjustment ratio of 10% to 50%; for the value trend of the adjusted proportionality coefficient α: small scene (fine-grained adjustment): when it is necessary to adjust invalid grids in a small area, the value of α is small (such as 0.1 to 0.2) to finely control the amplitude of grid increase; large scene (coarse-grained adjustment): when it is necessary to quickly reduce the number of invalid grids, the value of α is large (such as 0.3 to 0.5) to significantly increase the coverage of the grid.

[0171] For the value trend of the adjusted proportionality coefficient β: small scene (refined grid): when it is necessary to capture details with higher resolution, the value of β is small (such as 0.1 to 0.2) to slightly reduce the grid; large scene (rapid refinement): when it is necessary to quickly increase the grid resolution, the value of β is large (such as 0.3 to 0.5) to significantly reduce the grid size.

[0172] Meanwhile, a specific value example is provided: 1) In the scenario of vegetation monitoring (high-precision requirement), to accurately capture the spatial changes in vegetation distribution and reduce the grid cell size to improve the resolution, then α = 0.1 and β = 0.4; 2) In the scenario of disaster monitoring (high-coverage requirement), to quickly reduce invalid grids and increase the coverage of effective data, and increase the grid cell size, then α = 0.5 and β = 0.2; 3) In the scenario of urban building boundary monitoring (balanced requirement), both capturing the details of the building boundary and a certain coverage are needed, then α = 0.2 and β = 0.2; 4) In the scenario of water pollution monitoring (dynamic adjustment), when the water body range changes greatly, dynamically adjust the grid size to adapt to different regions, then α = 0.3 and β = 0.3. In short, the values of the adjusted proportionality coefficients α, β need to be combined with the specific usage scenario and input from the human-computer interaction interface, which is a well-known technical means in the art, so it will not be elaborated one by one in this embodiment.

[0173] Optionally, the feature extraction module includes a spectral feature extraction unit, a texture feature extraction unit, a shape feature extraction unit, and a multi-dimensional feature encoding unit. The spectral feature extraction unit analyzes the spectral information of the fused image. The texture feature extraction unit analyzes the texture characteristics of ground objects using the image gray-scale change information. The shape feature extraction unit analyzes the shape features of ground objects in the fused image based on edges and geometric structures. The multi-dimensional feature encoding unit integrates the extracted spectral, texture, and shape features into a unified multi-dimensional feature description matrix F(x,y), and performs normalization to form a normalized multi-dimensional feature description matrix F’(x,y).

[0174] The spectral feature extraction unit extracts the reflectance values of each band (red, green, near-infrared):

[0175] First, the multi-band data I k (x,y): k = 1, 2, …, K, representing the number of bands of the image; each band is a grayscale image, recording the reflectance value of the corresponding pixel in that band. The spectral feature of each pixel point is expressed as: S(x,y) = [B 1 (x,y), B 2 (x,y), …, B k (x,y)], where B k (x,y) is the value of the k-th band. For all pixels (x,y), the pixel values at the corresponding positions are read from each band image. The texture feature extraction unit is based on the statistical characteristics of the gray-level co-occurrence matrix (GLCM), that is, calculates the gray-level co-occurrence matrix: P(i,j,d,θ): representing the occurrence frequency P(i,j) of pixel pairs with gray values i and j at a distance d and in a direction θ. In this embodiment, the specific directions are: 0°, 45°, 90°, 135°; the distance d is 1 pixel or 2 pixels. Among them, the occurrence frequency P(i,j) of pixel pairs with gray values i and j is calculated according to the following formula:

[0176]

[0177] Among them, the number of occurrences is obtained through the grayscale image I(x,y), with a size of H×W and a pixel grayscale value range of [0, L-1], where L is the number of gray levels. The specified direction θ and distance d: direction θ: commonly 0°, 45°, 90°, 135°; distance d: usually takes 1 or 2 pixels, and counts the number of pixel pairs that meet the (i, j) condition. Then, traverse each pixel (x,y): check whether its neighboring pixel (x′,y′) is within the image range; if it is within the range, check (I(x,y) = i) ∧ (I(x′,y′) = j); if the condition is met, increment the count C(i,j) = C(i,j)+1, and output the count matrix C(i, j);

[0178] In this embodiment, the output count matrix C(i,j) is set to the number of occurrences; the determination of the total number of gray levels includes counting the number of pixel pairs that meet the conditions by traversing the image:

[0179] Traverse each pixel (x,y) in the grayscale image.

[0180] Determine whether the pixel (x,y) has a valid neighboring pixel (x′,y′): the neighboring pixel must be within the image range; if it is within the range, the pixel pair is considered valid. Accumulate the number of all valid pixel pairs, and if it is within the range, increment the count N pairs = N pairs + 1, and output N pairs . In this embodiment, N pairs is set to the total number of gray level pairs;

[0181] According to the gray-level co-occurrence matrix GLSM, determine the texture metrics of the image gray levels. The texture metrics include Contrast, Entropy, and Homogeneity. The specific metrics are calculated according to the following formula:

[0182]

[0183] In the formula, i,j are the indices of the gray levels, i,j ∈ [0,L - 1], where L is the number of gray levels (usually 256);

[0184] The shape feature extraction unit analyzes the connected regions in the edge map based on the binary result or edge detection result of the fused image, extracts the contour information of the ground objects, calculates the total number of pixels (area) and the total number of boundary pixels (perimeter) in the connected regions of the fused image, so as to obtain the shape factor ShapeFactor:

[0185]

[0186] In the formula, area is the area and perimeter is the perimeter; the multi-dimensional feature encoding unit combines the spectral, texture, and shape features of each pixel or region into a feature vector: F(x,y) = [S(x,y), T(x,y), C(x,y)]; in this embodiment, S(x,y) = [B 1 (x,y), B 2 (x,y), …, B k (x,y)], where B k (x,y) is the value of the k-th band, T(x,y) = [Contrast, Entropy, Homogeneity], and C(x,y) = [area, perimeter, ShapeFactor].

[0187] The multi-dimensional feature encoding unit normalizes the feature vector F(x,y) to obtain F’(x,y):

[0188]

[0189] In the formula, F min is the minimum value of all features in the feature vector F(x,y), F min = min(S min , T min , C min ), S min is the global minimum value of the spectral feature, T min is the global minimum value of the texture feature, C min is the global minimum value of the shape feature, F max is the maximum value of all features in the feature vector F(x,y), F max = max(S max , T max , C max ), S max is the global maximum value of the spectral feature, T max is the global maximum value of the texture feature, C max is the global maximum value of the shape feature.

[0190] For the global minimum value S min of the spectral feature and the global maximum value S max of the spectral feature, they are determined according to the following formula: S min = min(B k，min ), S max = max(B k，max ); specifically, traverse all bands k = 1, 2,..., K, and count the minimum value B k,min and the maximum value B k,max of each band.

[0191] For the global minimum value T min of the texture feature and the global maximum value T max of the texture feature, they are determined according to the following formula: T min = min(Contrast min , Entropy min , Homogeneity min ); T max = max(Contrast max , Entropy max , Homogeneity max ); that is, traverse the texture features of each pixel and count the minimum value and the maximum value of each dimension. For the global minimum value C minand the global maximum value C of the shape feature max , which is determined according to the following formula: C min = min(area min , perimeter min , ShapeFactor min ); C max = max(area max , perimeter max , ShapeFactor max ); that is, traverse the shape features of the ground object and directly count the minimum value C min and the maximum value C max .

[0192] Through the cooperation of the image fusion module and the feature extraction module, efficient feature analysis of the fused image is realized, and a multi-dimensional feature description matrix F(x, y) is extracted, making the anomaly detection of the evaluation module more accurate and ensuring the fluency and reliability of the data processing chain.

[0193] Optionally, the evaluation module obtains the multi-dimensional feature description matrix F’(x, y) and calculates the comprehensive anomaly index Abnormal(x, y) of the pixel point (x, y) according to the following formula:

[0194] In the formula, d is the number of feature dimensions of the standardized multi-dimensional feature description matrix F’(x, y). In this embodiment, d = K + d T + d C , that is, the total number of dimensions of the spectral feature, texture feature, and shape feature, Z i (x, y) is the standardized anomaly score of the pixel point (x, y) on the i-th feature dimension, CV i is the coefficient of variation of the i-th feature dimension, and α is an adjustment coefficient, the value of which is set by the system according to the actual situation;

[0195] The standardized anomaly score Z i (x, y) of the pixel point (x, y) on the i-th feature dimension is calculated according to the following formula:

[0196]

[0197] In the formula, F i ′(x, y) is the value of the i-th feature in the multi-dimensional feature description matrix, μ i is the global mean of the i-th feature dimension:

[0198] In the formula, W is the image width, H is the image height, and F i ′(x, y) is the value of the i-th feature in the multi-dimensional feature description matrix.

[0199] The global standard deviation σ of the i-th feature dimension i It is calculated according to the following formula:

[0200]

[0201] In the formula, W is the image width, H is the image height, and F i ′(x, y) is the value of the i-th feature in the multi-dimensional feature description matrix.

[0202] The coefficient of variation CV of the i-th feature dimension i It is calculated according to the following formula:

[0203]

[0204] In the formula, σ i is the global standard deviation of the i-th feature dimension, and μ i is the global mean of the i-th feature dimension.

[0205] In this embodiment, the value range of the adjustment coefficient α is: α ≥ 0, and a specific value example is provided: 1) In the scenario of vegetation anomaly detection in remote sensing images (detecting the vegetation health status in agricultural areas, discovering anomalies through spectral features (such as NDVI index), texture features may be affected by terrain changes, but are less significant for vegetation health), then α = 0.5; 2) In the scenario of abnormal building shape detection in urban planning (in urban remote sensing images, detecting abnormal building structures (such as overly slender or irregular shapes) through shape features (such as aspect ratio, shape factor), spectral features and texture features are only used as auxiliary information), then α = 2.0; 3) In the scenario of water area change monitoring (detecting changes in water area (such as lakes, rivers, etc.), spectral features are crucial, and shape features and texture features are secondary), then α = 0.8; In short, the specific value of the adjustment coefficient α needs to be set according to the actual scenario and input from the human-computer interaction interface, which is a well-known technical means in the art, so in this embodiment, it will not be elaborated one by one.

[0206] If the comprehensive anomaly index Abnormal(x, y) of the pixel point (x, y) does not fall within the evaluation threshold range [B1, B2] set by the system, it indicates that the pixel point (x, y) is abnormal. If the comprehensive anomaly index Abnormal(x, y) of the pixel point (x, y) falls within the evaluation threshold range [B1, B2] set by the system, it indicates that the pixel point (x, y) is not abnormal. The evaluation threshold range [B1, B2] set by the system is set by the system or the administrator according to the actual scenario and input through the human-computer interaction interface. This is a well-known technical means in the art, and those skilled in the art can query relevant technical manuals to obtain this technology. Therefore, it will not be elaborated one by one in this embodiment. Through the cooperation of the feature extraction module and the evaluation module, the efficient extraction and multi-dimensional comprehensive analysis of spectral features, texture features, and shape features are realized, enabling the system to evaluate the changes and anomalies in the monitoring area from multiple angles, ensuring the accuracy and comprehensiveness of anomaly detection, and enhancing the adaptability to complex scenarios.

[0207] The result output module includes an interaction unit and a warning unit. The interaction unit obtains the evaluation result and interacts with the warning unit. The warning unit issues a warning prompt about the evaluation result to the administrator. The interaction unit includes a data interface and an interaction logic processor. The data interface obtains the comprehensive anomaly index Abnormal(x, y) of the evaluation module. The interaction logic processor analyzes the relationship between the comprehensive anomaly index and the warning threshold range [B1, B2], marks the abnormal points or areas with different levels, and transmits them to the warning unit. Among them, the interaction logic processor conducts threshold comparison: if Abnormal(x, y) < B1, it is marked as a low-value anomaly; if Abnormal(x, y) > B2, it is marked as a high-value anomaly; if B1 ≤ Abnormal(x, y)

[0208] ≤ B2, it is marked as normal.

[0209] The warning unit includes a warning rule library, a warning generator, and a notification sender. The warning rule library stores warning rules corresponding to the anomaly levels (such as color identification, information level, notification method); the warning generator generates warning prompt information based on the data transmitted by the interaction unit, including graphical display and text description; the notification sender sends the warning information to the administrator through the set channels (such as text messages, emails, system notifications). For example: anomaly: red identification, send text message notification; normal: green identification, only system log record. During the warning notification process, in the visual interface, the abnormal area is highlighted: a heat map is displayed in the image in combination with the anomaly index matrix; different levels of abnormal areas are intuitively identified using color coding (red, green, etc.).

[0210] In addition, the present invention also provides a remote sensing image processing method, which includes the following steps: S1. Dynamically obtain satellite images and image data collected by drones in the monitoring area through a data acquisition module;

[0211] S2. Preprocess the satellite images and image data through an image fusion module to generate multi-spectral images and high-resolution images respectively, and perform fusion processing on the multi-spectral images and high-resolution images to form a fused image;

[0212] S3. Extract spectral features, texture features, and shape features from the fused image through a feature extraction module to generate a multi-dimensional feature description matrix;

[0213] S4. Evaluate the abnormal positions according to the multi-dimensional feature description data through an evaluation module to form an evaluation result;

[0214] S5. Generate a warning prompt according to the evaluation result through a result output module.

[0215] Optionally, the remote sensing image processing method further includes: in step S3, divide the multi-spectral image and high-resolution image generated after preprocessing through an image fusion module to generate grid cells of a fixed size.

[0216] Optionally, the remote sensing image processing method further includes: calculate the spectral consistency weight of the multi-spectral image in each grid cell through an analysis unit, evaluate the spatial detail richness of the high-resolution image in each grid cell, and perform fusion processing based on the spectral consistency weight and detail richness weight.

[0217] Through the cooperation of the evaluation module and the result output module, real-time detection and classification of abnormal positions are achieved, and through the linkage of the interaction unit and the warning unit, timely warning prompts are provided to the manager, ensuring the response speed and operation convenience of the system to potential risks.

[0218] Through the mutual cooperation of the data acquisition module, feature extraction module, image fusion module, evaluation module, and result output module, the remote sensing image processing system is efficient, accurate, and operable in multi-source data fusion, abnormal detection, and result output, ensuring that the entire system can analyze and warn the monitoring area in real time and comprehensively.

[0219] Embodiment 2: This embodiment should be understood as including all the features of any one of the foregoing embodiments and further improving on this basis. According to Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 and Figure 6As shown, the remote sensing image processing system further includes a change monitoring module. The change monitoring module obtains a multi-dimensional feature description matrix F′(x,y) standardized for one sampling period, calculates a change index Change(x,y), and triggers a change warning prompt. The change monitoring module includes a change analysis unit and a report generation unit. The change analysis unit analyzes the multi-dimensional feature description matrix based on the multi-dimensional feature description matrix F′(x,y) standardized for one sampling period to form an analysis result. The report generation unit gives a prompt to the manager according to the analysis result.

[0220] The change analysis unit obtains a multi-dimensional feature description matrix F′(x,y) standardized for one sampling period and calculates the change rate R of the i-th element according to the following formula i (x, y):

[0221]

[0222] In the formula, F′ i (x,y,t) is the value of the i-th element of the multi-dimensional feature description matrix standardized for the current period, and F′ i (x,y,t - 1) is the value of the i-th element of the multi-dimensional feature description matrix standardized for the previous period. ∈ is a small constant value, usually set as a very small positive number, for example, ∈ = 10 -6 ;

[0223] The change analysis unit obtains the change rate R i (x, y) and calculates the change index Change(x,y):

[0224]

[0225] In the formula, d is the number of feature dimensions of the standardized multi-dimensional feature description matrix F’(x,y), and R i (x, y) is the change rate of the i-th element.

[0226] If the change index Change(x,y) falls within the change warning threshold range [T low , T high set by the system, it indicates that the change amount is within the acceptable range and no alarm is required. If the change index Change(x,y) does not fall within the change warning threshold range [T low , T high set by the system, it indicates that the change amount is abnormal and an alarm needs to be triggered. Specifically, if Change(x,y) < T low , it is in a low-value abnormal state, indicating that the change amount is too small, which may reflect inaccurate data or small changes; if Change(x,y) > T high , it is in a high-value abnormal state, indicating that the change amount is too large, which may reflect significant changes or risks in the monitored object.

[0227] The report generation unit includes a data retriever and a report generator. The data retriever retrieves the analysis results of the change analysis unit (the relationship between the change index Change(x, y) and the system-set change warning threshold range [T low , T high ), and the report generator generates corresponding text descriptions based on the comparison results of the data retriever and gives prompts to the manager;

[0228] In this embodiment, an example of the text description is as Figure 5 shown; at the same time, through the cooperation between the change monitoring module and the feature extraction module, the system can effectively achieve a comprehensive analysis and abnormal warning of the target area based on the accurate extraction of multi-dimensional features and the real-time monitoring of dynamic changes, ensuring that the entire system has the advantages of high precision, high real-time performance, and intelligence.

[0229] The content disclosed above is only the preferred feasible embodiment of the present invention, and does not limit the protection scope of the present invention. Therefore, all equivalent technical changes made by using the content of the specification and drawings of the present invention are included in the protection scope of the present invention. In addition, the elements therein can be updated with the development of technology.

Claims

1. A remote sensing image processing system, comprising an unmanned aerial vehicle, characterized in that: The remote sensing image processing system also includes a data acquisition module, a feature extraction module, an image fusion module, an evaluation module and a result output module; The data acquisition module dynamically acquires satellite images of the monitored area and image data acquired by the drone, the image fusion module pre-processes the satellite images and image data to generate a multispectral image and a high-resolution image respectively, and fuses the multispectral image and the high-resolution image to form a fused image, the feature extraction module extracts spectral features, texture features and shape features from the fused image to generate a matrix containing multidimensional feature descriptions, the evaluation module evaluates the abnormal position according to the multidimensional feature description data to form an evaluation result, and the result output module generates an early warning prompt according to the evaluation result; The image fusion module includes a preprocessing unit, a grid division unit, an analysis unit, a weighted fusion unit and a fusion optimization unit. The preprocessing unit preprocesses satellite images and image data to generate multispectral images and high-resolution images respectively. The grid division unit divides the input multispectral image and high-resolution image into grid units of fixed size. The analysis unit calculates the spectral consistency weight of the multispectral image in each grid unit and evaluates the spatial detail richness weight of the high-resolution image in each grid unit. The weighted fusion unit performs fusion processing based on the spectral consistency weight and the detail richness weight. The fusion optimization unit performs quality evaluation and optimization on the preliminary fused image to form a fused image. The fusion optimization unit evaluates the information volume and quantitative evaluation index of the preliminary fusion image, and if the index does not meet the monitoring threshold of the system, the size and weight of the grid unit are readjusted.

2. A remote sensing image processing system according to claim 1, characterized in that: The data acquisition module includes an image acquisition unit and an image acquisition unit. The image acquisition unit retrieves satellite image data of the monitored area, and the image acquisition unit collects image data collected by a drone. The image acquisition unit is loaded on the drone and follows the drone to collect image data of the monitored area.

3. A remote sensing image processing system according to claim 2, characterized in that: The grid division unit includes a spatial alignment component, a grid generation component, a multi-source mapping grid mapping component and a grid quality control component. The spatial alignment component aligns the spatial coordinate systems of the multispectral image and the high-resolution image. The grid generation component generates a fixed-size grid unit according to the spatial range and resolution of the image. The multi-source mapping grid mapping component maps the pixel values ​​of the multispectral image and the high-resolution image to the corresponding grid units. The grid quality control component evaluates the quality evaluation result of the grid division and adjusts the grid size or range according to the evaluation result.

4. A remote sensing image processing system according to claim 3, characterized in that: The analysis unit includes a spectral consistency component, a spatial detail evaluation component and a weight normalization component. The spectral consistency component analyzes the spectral distribution characteristics of the multispectral image in each grid unit and calculates the spectral consistency weight. The spatial detail evaluation component evaluates the detail richness of the high-resolution image in each grid unit. The weight normalization component normalizes the spectral consistency weight and the detail richness weight.

5. A remote sensing image processing system according to claim 4, characterized in that: The weighted fusion unit includes a weight mapper, a pixel fusion unit, a boundary smoothing component and a global adjustment component. The weight mapper receives the spectral consistency weight and detail richness weight calculated by the analysis unit and maps them to the pixel level of each grid unit. The pixel fusion unit performs weighted superposition on the pixels in each grid unit to generate a preliminary fused image. The boundary smoothing component smoothes the boundaries between grid units. The global adjustment component performs global adjustment on the fused preliminary image.

6. A remote sensing image processing system according to claim 5, characterized in that: The feature extraction module includes a spectral feature extraction unit, a texture feature extraction unit, a shape feature extraction unit and a multidimensional feature encoding unit. The spectral feature extraction unit analyzes the spectral information of the fused image, the texture feature extraction unit uses the image grayscale change information to analyze the texture characteristics of the object, the shape feature extraction unit analyzes the shape characteristics of the object in the fused image based on the edge and geometric structure, and the multidimensional feature encoding unit integrates the extracted spectral, texture and shape features into a unified multidimensional feature description matrix F(x,y), and standardizes them to form a standardized multidimensional feature description matrix F'(x,y).

7. A remote sensing image processing system according to claim 6, characterized in that: The evaluation module describes the multidimensional feature matrix F'(x, y) and calculates the comprehensive abnormal index Abnormal(x, y) of the pixel point (x, y) according to the following formula: ； Where d is the feature dimension of the standardized multidimensional feature description matrix F'(x,y), Z i (x, y) is the standardized anomaly score of the pixel (x, y) in the i-th feature dimension, CV i is the coefficient of variation of the i-th feature dimension, α is the adjustment coefficient, and its value is set by the system according to the actual situation; If the comprehensive abnormal index Abnormal (x, y) of the pixel point (x, y) does not fall within the evaluation threshold range [B1, B2] set by the system, it means that the pixel point (x, y) is abnormal.

8. A remote sensing image processing method, applied to a remote sensing image processing system as claimed in claim 7, characterized in that: The remote sensing image processing method comprises the following steps: S1. Dynamically acquire satellite images of the monitored area and image data collected by the drone through a data acquisition module; S2, pre-processing the satellite image and image data through the image fusion module to generate a multispectral image and a high-resolution image respectively, and fusing the multispectral image and the high-resolution image to form a fused image; S3, extracting spectral features, texture features and shape features from the fused image through a feature extraction module to generate a multi-dimensional feature description matrix; S4, evaluating the abnormal position according to the multi-dimensional feature description data through the evaluation module to form an evaluation result; S5. Generate early warning prompts according to the evaluation results through the result output module.

9. A remote sensing image processing method according to claim 8, characterized in that: The remote sensing image processing method further includes: in step S3, dividing the multispectral image and the high-resolution image generated after preprocessing by an image fusion module to generate grid units of fixed size.

10. A remote sensing image processing method according to claim 9, characterized in that: The remote sensing image processing method also includes: calculating the spectral consistency weight of the multispectral image in each grid unit through an analysis unit, evaluating the spatial detail richness weight of the high-resolution image in each grid unit, and performing fusion processing based on the spectral consistency weight and the detail richness weight.

Images