Remote sensing image processing system and processing method

Abstract

The invention relates to the technical field of image extraction, and provides a remote sensing image processing system, which comprises a server, an unmanned aerial vehicle, a data acquisition module, a feature extraction module, an image fusion module, an evaluation module and a result output module, the image fusion module preprocesses the satellite image and the image data to generate a multispectral image and a high-resolution image respectively, and performs fusion processing on the multispectral image and the high-resolution image to form a fused image, and the feature extraction module extracts spectral features, texture features and shape features from the fused image; the evaluation module evaluates the abnormal position according to the multi-dimensional feature description data to form an evaluation result, and the result output module generates an early warning prompt according to the evaluation result.

Description

Technical Field

[0001] The present invention relates to the technical field of image extraction, and in particular to a remote sensing image processing system and a processing method. Background Art

[0002] Remote sensing refers to the image data acquisition technology that detects and records the radiation and reflection characteristics of the target's electromagnetic waves through hyperspectral cameras and area array cameras without contacting the target.

[0003] For example, Chinese patent CN101576994B discloses a remote sensing image processing method and device, which selects sub-nodes with strong computing power to participate in the processing of remote sensing images according to the maximum data processing amount of the sub-nodes in the grid, thereby improving the processing speed of the remote sensing images; by dividing the remote sensing image into multiple remote sensing sub-images, the sub-nodes in the grid can realize parallel processing of the remote sensing sub-images according to the processing capabilities of the sub-nodes. Although this improves the processing speed, it ignores the consideration of multi-source data fusion capabilities, resulting in a significant reduction in the reliability of image analysis.

[0004] The present invention is made to solve the common problems in this field, such as low processing efficiency, large loss of spectral information and spatial details, poor multi-source data fusion capability, low intelligence, poor evaluation means, poor classification and change detection accuracy, and insufficient visibility of analysis results. Summary of the invention

[0005] The purpose of the present invention is to propose a remote sensing image processing system and a processing method in view of the current deficiencies.

[0006] In order to overcome the shortcomings of the prior art, the present invention adopts the following technical solutions:

[0007] A remote sensing image processing system comprises a server and a drone. The remote sensing image processing system also comprises a data acquisition module, a feature extraction module, an image fusion module, an evaluation module and a result output module. The server is respectively connected to the data acquisition module, the feature extraction module, the image fusion module, the evaluation module and the result output module.

[0008] The data acquisition module dynamically acquires satellite images of the monitored area and image data acquired by the drone; the image fusion module pre-processes the satellite images and image data to generate a multispectral image and a high-resolution image respectively, and fuses the multispectral image and the high-resolution image to form a fused image; the feature extraction module extracts spectral features, texture features and shape features from the fused image to generate a matrix containing multidimensional feature descriptions; the evaluation module evaluates the abnormal position according to the multidimensional feature description data to form an evaluation result; and the result output module generates an early warning prompt according to the evaluation result.

[0009] The image fusion module includes a preprocessing unit, a grid division unit, an analysis unit, a weighted fusion unit and a fusion optimization unit. The preprocessing unit preprocesses the satellite image and image data to generate a multispectral image and a high-resolution image respectively. The grid division unit divides the input multispectral image and high-resolution image into grid units of fixed size. The analysis unit calculates the spectral consistency weight of the multispectral image in each grid unit and evaluates the spatial detail richness of the high-resolution image in each grid unit. The weighted fusion unit performs fusion processing based on the spectral consistency weight and the detail richness weight. The fusion optimization unit performs quality assessment and optimization on the preliminary fused image to form a fused image.

[0010] The fusion optimization unit evaluates the information volume and quantitative evaluation index of the preliminary fusion image, and if the index does not meet the monitoring threshold of the system, the size and weight of the grid unit are readjusted.

[0011] Optionally, the data acquisition module includes an image acquisition unit and an image acquisition unit, the image acquisition unit retrieves satellite image data of the monitored area, the image acquisition unit collects image data collected by a drone, the image acquisition unit is loaded on the drone, and follows the drone to collect image data of the monitored area.

[0012] Optionally, the grid division unit includes a spatial alignment component, a grid generation component, a multi-source mapping grid mapping component and a grid quality control component, the spatial alignment component aligns the spatial coordinate systems of the multispectral image and the high-resolution image, the grid generation component generates a fixed-size grid unit according to the spatial range and resolution of the image, the multi-source mapping grid mapping component maps the pixel values ​​of the multispectral image and the high-resolution image to corresponding grid cells, and the grid quality control component evaluates the quality assessment result of the grid division and adjusts the grid size or range according to the assessment result.

[0013] Optionally, the analysis unit includes a spectral consistency component, a spatial detail evaluation component and a weight normalization component, the spectral consistency component analyzes the spectral distribution characteristics of the multispectral image in each grid unit and calculates the spectral consistency weight, the spatial detail evaluation component evaluates the detail richness of the high-resolution image in each grid unit, and the weight normalization component normalizes the spectral consistency weight and the detail richness weight.

[0014] Optionally, the weighted fusion unit includes a weight mapper, a pixel fusion unit, a boundary smoothing component and a global adjustment component, the weight mapper receives the spectral consistency weight and detail richness weight calculated by the analysis unit and maps them to the pixel level of each grid unit, and the pixel fusion unit performs weighted superposition on the pixels in each grid unit to generate a preliminary fused image.

[0015] The boundary smoothing component smoothes the boundaries between grid cells, and the global adjustment component performs global adjustment on the fused preliminary image.

[0016] Optionally, the feature extraction module includes a spectral feature extraction unit, a texture feature extraction unit, a shape feature extraction unit and a multidimensional feature encoding unit. The spectral feature extraction unit analyzes the spectral information of the fused image, the texture feature extraction unit uses the image grayscale change information to analyze the texture characteristics of the object, the shape feature extraction unit analyzes the shape characteristics of the object in the fused image based on the edge and geometric structure, and the multidimensional feature encoding unit integrates the extracted spectral, texture and shape features into a unified multidimensional feature description matrix F(x,y), and standardizes them to form a standardized multidimensional feature description matrix F'(x,y).

[0017] Optionally, the evaluation module describes the matrix F'(x, y) based on the multidimensional feature and calculates the comprehensive abnormal index Abnormal(x, y) of the pixel point (x, y) according to the following formula:

[0018]

[0019] Where d is the feature dimension of the standardized multidimensional feature description matrix F'(x,y), Z i (x, y) is the standardized anomaly score of the pixel (x, y) in the i-th feature dimension, CV i is the coefficient of variation of the i-th feature dimension, α is the adjustment coefficient, and its value is set by the system according to the actual situation;

[0020] If the comprehensive abnormal index Abnormal(x, y) of the pixel point (x, y) does not fall within the evaluation threshold range [B1, B2] set by the system, it means that the pixel point (x, y) is abnormal.

[0021] In addition, the present invention also provides a remote sensing image processing method, which comprises the following steps:

[0022] S1. Dynamically acquire satellite images of the monitored area and image data collected by the drone through a data acquisition module;

[0023] S2, pre-processing the satellite image and image data through the image fusion module to generate a multispectral image and a high-resolution image respectively, and fusing the multispectral image and the high-resolution image to form a fused image;

[0024] S3, extracting spectral features, texture features and shape features from the fused image through a feature extraction module to generate a multi-dimensional feature description matrix;

[0025] S4, evaluating the abnormal position according to the multi-dimensional feature description data through the evaluation module to form an evaluation result;

[0026] S5. Generate early warning prompts according to the evaluation results through the result output module.

[0027] The beneficial effect achieved by the present invention is that through the mutual cooperation of the data acquisition module, the feature extraction module, the image fusion module, the evaluation module and the result output module, the remote sensing image processing system has high efficiency, accuracy and operability in multi-source data fusion, anomaly detection and result output, ensuring that the entire system can perform efficient analysis and early warning of the monitored area in real time and comprehensively. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate the same parts.

[0029] Figure 1 It is an overall block diagram of the present invention.

[0030] Figure 2 It is a block diagram of the image fusion module of the present invention.

[0031] Figure 3 Schematic diagram of the extraction block of the feature extraction module of the present invention.

[0032] Figure 4 The figure is a flow chart of the remote sensing image processing method of the present invention.

[0033] Figure 5 This is a schematic diagram of a report generated for a certain area by the report generating unit of the present invention.

[0034] Figure 6 It is a schematic diagram of the effects of the prior art and the technology of the present application. DETAILED DESCRIPTION

[0035] The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the protection scope of the present invention.

[0036] Embodiment 1: According to Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 As shown, this embodiment provides a remote sensing image processing system, which includes a server and a drone. The remote sensing image processing system also includes a data acquisition module, a feature extraction module, an image fusion module, an evaluation module and a result output module. The server is connected to the data acquisition module, the feature extraction module, the image fusion module, the evaluation module and the result output module respectively and stores the intermediate data and process data of the data acquisition module, the feature extraction module, the image fusion module, the evaluation module and the result output module in the database of the server for query and call. The data acquisition module dynamically acquires satellite images of the monitored area and image data acquired by drones, the image fusion module pre-processes the satellite images and image data to generate multispectral images and high-resolution images respectively, and fuses the multispectral images and high-resolution images to form a fused image, the feature extraction module extracts spectral features, texture features and shape features from the fused image to generate a multi-dimensional feature description matrix, the evaluation module evaluates the abnormal position according to the multi-dimensional feature description data to form an evaluation result, and the result output module generates an early warning prompt according to the evaluation result. In this embodiment, the remote sensing image processing system also includes a central processing unit, which is respectively controlled and connected to the data acquisition module, the feature extraction module, the image fusion module, the evaluation module and the result output module, and centrally controls the data acquisition module, the feature extraction module, the image fusion module, the evaluation module and the result output module based on the central processing unit, and stores the control data of the central processing unit in the server to ensure the reliability and accuracy of the remote sensing image processing of the entire system.

[0037] The image fusion module includes a preprocessing unit, a grid division unit, an analysis unit, a weighted fusion unit and a fusion optimization unit. The preprocessing unit preprocesses the satellite image and image data to generate a multispectral image and a high-resolution image respectively. The grid division unit divides the input multispectral image and high-resolution image into grid units of fixed size. The analysis unit calculates the spectral consistency weight of the multispectral image in each grid unit and evaluates the spatial detail richness of the high-resolution image in each grid unit. The weighted fusion unit performs fusion processing based on the spectral consistency weight and the detail richness weight. The fusion optimization unit performs quality assessment and optimization on the preliminary fusion image to form a fusion image. Among them, the fusion optimization unit evaluates the information volume and quantitative evaluation index of the preliminary fusion image. If the index does not meet the monitoring threshold of the system, the size and weight of the grid unit are readjusted.

[0038] Optionally, the data acquisition module includes an image acquisition unit and an image acquisition unit, the image acquisition unit retrieves the satellite image data of the monitoring area, the image acquisition unit acquires the image data acquired by the drone, the image acquisition unit is loaded on the drone, and follows the drone to acquire the image data of the monitoring area. The image acquisition unit includes a call commander and a data receiver, the call commander sends a call command to the satellite, the call command includes the geographical boundary, time range and resolution requirement of the monitoring area; the data receiver receives the satellite image data fed back by the satellite. In this embodiment, the satellite image data fed back by the satellite includes the following contents: data timestamp, satellite name (such as Sentinel-2), spatial resolution and spectral range of available images, and image download link. The image acquisition unit includes a data storage device and an image acquisition probe, the image acquisition probe acquires the image data of the monitoring area, and the data storage device stores the image data of the monitoring area acquired by the image acquisition probe, wherein, in the process of acquiring the image data of the monitoring area, the drone is controlled to patrol the set monitoring area to obtain the image data of the monitoring area; at the same time, the monitoring area patrolled by the drone must be consistent with the geographical boundary or monitoring position called by the satellite.

[0039] In this embodiment, how to control the drone to perform inspections in the monitoring area is a conventional technical means, and therefore, it will not be described in detail in this embodiment.

[0040] The steps of preprocessing the satellite images and image data collected by the UAV by the preprocessing unit include:

[0041] S100, input satellite imagery and drone imagery data:

[0042] S101, respectively performing geometric correction, radiation correction and cropping processing on satellite image and UAV image data;

[0043] Among them, geometric correction, radiation correction and cropping of satellite imagery and drone image data is an image processing technology, and is a technical means well known to technicians in this field.

[0044] Optionally, the raster division unit includes a spatial alignment component, a raster generation component, a multi-source mapping raster mapping component and a raster quality control component. The spatial alignment component aligns the spatial coordinate systems of the multispectral image and the high-resolution image. The raster generation component generates fixed-size raster cells according to the spatial range and resolution of the image. The multi-source mapping raster mapping component maps the pixel values ​​of the multispectral image and the high-resolution image to the corresponding raster cells. The raster quality control component evaluates the quality assessment results of the raster division and adjusts the raster size or range according to the assessment results. Evaluate the coverage of the raster division. If there is insufficient valid data in some raster cells, readjust the raster size or range. The spatial alignment component aligns the spatial coordinate systems of the multispectral image and the high-resolution image to ensure that the two images have the same geographic reference system (CRS, Coordinate Reference System) to facilitate subsequent raster division and data mapping.

[0045] The geographic boundaries of multispectral images and high-resolution images are expressed as follows: Multispectral image boundary: [x min 多 ,x max 多 ,y min 多 ,y max 多 ]. High resolution image boundary: [x min 高 ,x max 高 ,y min 高 ,y max 高 ].

[0046] Overlapping part (intersection): x min =max(x min 多 , x min 高 ), x max =min(x max 多 , x max 高 );y min =max(y min 多 ,y min 高 ), y max =min(y max 多 ,y max 高 );

[0047] Completely covered part (union): xmin =min(x min 多 , x min 高 ), x max =max(x max 多 , x max 高 );

[0048] y min =min(y min 多 ,y min 高 ), y max =max(y max 多 ,y max 高 );

[0049] The grid generation component determines the grid unit size: Set the size of the grid unit s: s = R t *k; where R t is the image resolution, k is the scaling factor (default is 1), the value range is [0.1, 10], or according to the scene setting,

[0050] In this embodiment, an example of the value of the scaling factor k is provided: in the scenario of large-scale land use classification, the scaling factor k=10; in the scenario of vegetation monitoring in precision agriculture, the scaling factor k=1; in the scenario of urban heat island effect analysis, the scaling factor k=4; in the scenario of refined monitoring of natural disasters, the scaling factor k=0.5; in short, different scaling factors are selected for different regional analyses, for example: large regional analysis: k>1, the grid unit becomes larger; refined analysis: k<1, the grid unit becomes smaller. The grid generation component generates a grid network according to the following formula:

[0051]

[0052] Each grid cell is numbered G. ij , i,j are row and column indices, To round down the sign, the multi-source mapping grid mapping component assigns pixels to grid cells: the grid generation component calculates the grid cell G to which each pixel point (x, y) belongs. ij :

[0053]

[0054] in, is the rounding symbol, i, j are the row and column indices; the grid generation component generates each grid cell G ij, calculate the statistical characteristics of the included multispectral pixel values ​​and high-resolution pixel values:

[0055]

[0056] In the formula, F ij 多光谱 is the grid unit G ij The average pixel value of the multispectral image in F ij 高分辨率 is the grid unit G ij The average pixel value of the multispectral image in is, N and M are the number of multispectral pixels and high-resolution pixels in the grid, respectively, k 多光谱 is the grid unit G ij The value of the kth multispectral pixel in I k 高分辨率 is the grid unit G ij The value of the kth high-resolution pixel in . When the characteristic value F of each grid cell is calculated ij 多光谱 and F ij 高分辨率 Then, store it.

[0057] In the process of evaluating the quality of grid division by the grid quality control component, each grid unit G is calculated according to the following formula ij , calculate the effective data pixel ratio C ij :

[0058] Among them, N 有效 is the number of valid data pixels in the grid cell, N 总 is the total number of pixels in the grid cell, and its value is calculated according to the following formula:

[0059] Where s is the side length of the grid cell (unit: meter), R is the resolution of the image (unit: meter / pixel), and R 2 is the area of ​​a single pixel.

[0060] The valid pixel is determined by traversing all pixel values ​​in the grid cell I k , count the number of pixels that meet the valid conditions:

[0061] Where Ⅱ(·) is the indicator function: Among them, I k is the pixel value of the kth pixel in the grid cell.

[0062] Invalid values ​​refer to pixel values ​​that cannot be used for effective calculation and analysis. In this embodiment, abnormal invalid values ​​are identified by statistical methods, that is, the mean μ' and standard deviation σ' of the pixel values ​​are calculated, and pixels that deviate from the mean by more than a threshold value (such as 3σ') are regarded as invalid values;

[0063] The process of determining and adjusting the validity of the grid unit includes: Step 1: Determine whether the grid unit is valid: For example, the effective data pixel ratio C ij Less than the coverage threshold T set by the system 覆盖度 , then the grid cell is considered invalid; that is, if there is insufficient valid data in a grid (such as fewer valid pixels or too many invalid values), the grid cell will be marked as invalid. Step 2: Count the proportion of invalid grids; if the proportion of invalid grids exceeds the set threshold R 无效 , adjust the grid cell size, where the new grid size s 新 Calculate according to the following formula: 新 =s·f,f∈[0.5,2]; where s is the original grid size;

[0064] Where f is the adjustment factor. The grid size is increased according to insufficient coverage, or reduced according to over-dense coverage. After the grid cell size is adjusted, the grid is regenerated and mapped.

[0065] The analysis unit includes a spectral consistency component, a spatial detail evaluation component and a weight normalization component. The spectral consistency component analyzes the spectral distribution characteristics of the multispectral image in each grid unit and calculates the spectral consistency weight. The spatial detail evaluation component evaluates the detail richness of the high-resolution image in each grid unit. The weight normalization component normalizes the spectral consistency weight and the detail richness weight.

[0066] The spectral consistency component analyzes the spectral distribution characteristics of the multispectral image in each grid cell, and the process of calculating the spectral consistency weight includes: S300, extracting the multispectral data in the grid cell.

[0067] S301, calculate the spectrum mean μ and standard deviation σ:

[0068] Where, X i It represents the average value of all pixel values ​​of the grid cell in the i-th band of the multispectral image, that is, the mean value of the pixel values ​​of the i-th band in the grid cell, and n is the number of bands.

[0069] S303, calculate the coefficient of variation CV as a spectrum consistency index:

[0070] S304, calculating the spectral consistency weight ω 光谱,ij :ω 光谱,ij=1-CV; in this example, the smaller the coefficient of variation CV, the higher the spectral consistency and the greater the weight

[0071] The evaluation process of the spatial detail evaluation component includes: 1) extracting high-resolution image data within the grid cells; 2) extracting image edges using edge detection algorithms (such as Sobel or Canny operators).

[0072] In this example, the edge detection formula (taking Sobel as an example) is:

[0073]

[0074] Where I(x,y) is the gray value matrix within the grid unit, which represents the gray intensity of each pixel within the grid range, G x and G y Respectively represent the gradient of the image in the x and y directions, and G is the edge intensity. Among them, edge detection (such as the Sobel operator) is an algorithm that extracts edge features based on the change of image gradient. The grayscale value can directly reflect the intensity change between pixels, so it is necessary to convert the high-resolution image data into a grayscale image for processing. For color images (such as RGB images), they need to be converted into grayscale images first:

[0075] I(x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where R(x,y), G(x,y), and B(x,y) are the red, green, and blue channel values ​​of the pixel, respectively.

[0076] 3) Calculate edge density D 边缘 :

[0077] Where, the number of edge pixels is the number of pixels in the grid unit that are identified as edges, N 总 is the total number of pixels in the grid unit. The number of edge pixels is calculated by edge detection algorithm to calculate the gradient strength of pixels in the grid unit. Specifically, the Sobel operator is used, and when G k >T 边缘 , then mark the pixel as an edge pixel. k is the edge strength of the kth pixel in the grid unit, and its value is calculated by the Sobel operator;

[0078] In addition, T 边缘 is the threshold of edge intensity, which is used to distinguish edge pixels from non-edge pixels. Its value is set by the system, that is, a fixed value is set in advance. For example, T is set according to experimental experience. 边缘 =50 (the intensity range is usually 0 to 255). Count the number of edge pixels: traverse all pixels in the grid unit and count the number of pixels that meet the conditions:

[0079] Where Ⅱ(·) is the indicator function:

[0080] 4) D 边缘 Represents the detail richness weight ω 细节,ij ; Weight normalization component spectral consistency weight ω for all grid cells 光谱,ij and detail richness weight ω 细节,ij Perform normalization:

[0081]

[0082] In the formula, ω′ 光谱,ij is the normalized spectral weight, ω′ 细节,ij is the normalized detail weight, w 光谱,ij is the spectral consistency weight, w 细节,ij is the detail richness weight, w 光谱，min is the minimum value of the spectral consistency weight, w 光谱，max is the maximum value of the spectral consistency weight, w 细节，min is the minimum value of the spectral consistency weight, w 细节，max is the maximum value of the spectral consistency weight; the normalized weight ω′ of each grid cell 光谱,ij and ω′ 细节,ij Output to the weighted fusion unit.

[0083] Optionally, the weighted fusion unit includes a weight mapper, a pixel fuser, a boundary smoothing component and a global adjustment component. The weight mapper receives the spectral consistency weight and detail richness weight calculated by the analysis unit and maps them to the pixel level of each grid unit. The pixel fuser performs weighted superposition on the pixels within each grid unit to generate a preliminary fused image. The boundary smoothing component smoothes the boundaries between grid units to eliminate image discontinuity problems caused by grid division. The global adjustment component performs global adjustments on the fused preliminary image to ensure a balance between overall spectral information and detail information.

[0084] The weight mapper maps the spectral consistency weight and detail richness weight to the pixel level of each grid unit according to the following steps: S400, receiving the spectral consistency weight and detail richness weight calculated by the analysis unit, traversing all grid units during the allocation process, and storing the normalized weight ω′ 光谱,ij and ω′ 细节,ij ;

[0085] S401, determine the pixel range of each grid unit: according to the grid division size s and the image resolution R, determine the pixel range of each grid unit G ij The number of pixels N in:

[0086] Assume that the grid cell G ij The boundary range is: row range = [i start ,i end ], column range = [j start ,j end ];

[0087] S402, assigning a uniform weight value: for each grid unit G ij , the normalized spectral weight ω′ 光谱,ij and the normalized detail weight ω′ 细节,ij Assigned to all pixels within this grid cell:

[0088] ω′ 光谱,k =ω′ 光谱,ij ,ω′ 细节,k =ω′ 细节,ij ,k∈G ij ;

[0089] In the formula, ω′ 光谱,k is the pixel-level spectral weight, ω′ 细节,k is the pixel-level detail weight;

[0090] In the above formula, the meaning of the whole formula is: assign the grid-level spectral weight and detail weight to all pixels in the grid; specifically, each grid unit G ij It is determined by the grid size s and the image resolution R; the grid division ensures that each ij corresponds to a unique spatial region. start and i end Indicates the starting and ending points of the pixel row of the grid; j start and j end Indicates the starting and ending points of the pixel columns of the grid; each grid unit G ij All pixels k are determined. Therefore, the grid-level weight ω′ 光谱,ij and ω′ 细节,ij Directly assigned to G ij In other words, each pixel k can be uniquely mapped to a grid cell G according to its row and column coordinates. ij .

[0091] S403, storing the mapped pixel weights: recording the spectrum and detail weight value ω′ for each pixel k 光谱,k and ω′ 细节,k .

[0092] S404, generating pixel-level weight output: storing the weight of each pixel in an image data format to facilitate fusion with multispectral and high-resolution images.

[0093] Create two weight maps with the same resolution as the original image: spectral weight map W spec :The value of each pixel is ω′ 光谱,k ; Detail weight map W detail :The value of each pixel is ω′ 细节,k ;

[0094] Among them, the spectral weight map W spec is a matrix consistent with the resolution of the input image, where the value of each pixel is equal to the normalized spectral consistency weight ω′ of the grid cell where the pixel is located. 光谱,k :

[0095] W spec (x,y)=ω′ 光谱，k ,k∈G ij ;

[0096] In the formula, (x, y) represents the position of the pixel in the full image coordinates, k is the index of the pixel, and G ij is the grid cell where the pixel is located.

[0097] Detail weight map W detail It is a matrix consistent with the resolution of the input image, in which the value of each pixel is equal to the normalized detail richness weight ω′ of the grid cell where the pixel is located 细节,k :W detail (x,y)=ω′ 细节,k ,k∈G ij ;

[0098] In the formula, (x, y) represents the position of the pixel in the full image coordinates, k is the index of the pixel, and G ij is the grid cell where the pixel is located.

[0099] The process of generating a weight graph includes the following steps:

[0100] S1000, input data: grid-level weight matrix: ω′ 光谱,ij is the normalized spectral weight for each grid cell; ω′ 细节,ij The normalized detail weight of each grid cell. The size of the grid division: the side length of the grid cell s; the resolution R of the input image.

[0101] S1001, mapping to pixel level: determining the pixel range corresponding to the grid unit: each grid unit G ij The pixel row and column range included:

[0102] For example, if the grid size s = 100m and the image resolution R = 10m, then the pixel range of each grid is 10×10. Assign a uniform weight value: Assign the grid-level weight ω′ 光谱,ij and ω′细节,ij Assigned to all pixels within that grid cell.

[0103] S1002, storing weight map: spectral weight map W spec :At each pixel position (x, y), fill in the corresponding spectral weight value. Detail weight map W detail : At each pixel position (x, y), fill in the corresponding detail weight value.

[0104] The pixel fusion unit performs weighted calculation on the pixel values ​​of the multispectral image and the high-resolution image according to the spectral consistency weight and the detail richness weight. Specifically, read the multispectral pixel I k 多光谱 and high resolution pixel I k 高分辨率 , and the spectral weight map W spec (x,y), detail weight map W detail (x,y);

[0105] For each pixel (x, y), the fusion is performed according to the following formula:

[0106]

[0107] According to the above formula, the fusion I of each pixel (x, y) is fused 融合 (x,y), the fused pixel value I 融合 (x,y) constitutes the initial fusion image I 初步融合 .

[0108] The boundary smoothing component smoothes the boundary areas between grid cells to eliminate the image discontinuity problem caused by grid division and make the image transition more natural. The specific steps include: 1) Detecting the boundary area: marking the boundary area of ​​each grid cell, that is, through the grid index G ij The row and column range of: row range = [i start ,i end ]; column range = [j start ,j end ]; mark the pixels in the boundary area as the transition area. 2) Smoothing: Use Gaussian blur convolution kernel to smooth the boundary area:

[0109]

[0110] Where: K(m,n) is the Gaussian kernel: m and n are the row and column distances of pixel offset;

[0111] The size of the kernel is usually determined by a range of 6σ (i.e., the kernel boundary is 3 times σ away from the center), σ is a parameter of the blur degree, and its value range is: (0, ∞), σ→0: indicates no blur, the image remains completely clear, σ→∞: the image is extremely blurred and tends to a single color. In this embodiment, σ is usually taken in the range of: [0.5, 10]; lower σ values ​​(such as 0.5-2): used for subtle blur or noise suppression; higher σ values ​​(such as 5-10): used for smoothing of larger areas or boundary transitions.

[0112] The global adjustment component performs global adjustment on the fused preliminary image. The steps to ensure the overall balance of spectral information and detail information include:

[0113] S500, spectral equalization: spectral adjustment is performed on the smoothed image to make the spectral information consistent with the distribution of the multi-spectral image: I 光谱 (x,y)=I 平滑 (x,y)+λ·(I 多光谱 (x,y)-I 平滑 (x,y));

[0114] Where λ is the spectrum adjustment coefficient, which is set by the system and has a range of [0, 1], and is used to balance the spectrum and detail information. 光谱 (x, y) is the pixel value after spectral equalization, I 平滑 (x,y) is the initial fusion image I 初步融合 (x, y) is the image pixel value after being processed by the boundary smoothing component, I 多光谱 (x, y) is the original pixel value of the multispectral image at pixel (x, y); where λ = 0: the smoothing image I is completely adopted 平滑 The value of does not consider the characteristics of the multispectral image; λ = 1: completely adjust according to the spectral characteristics of the multispectral image, ignoring the influence of the smoothed image;

[0115] S501, detail enhancement: extract the detail information (such as edges and textures) of the high-resolution image and superimpose it on the spectrally adjusted image: I 增强 (x,y)=I 光谱 (x,y)+β·ΔI 高分辨率 (x,y);

[0116] In the formula, I 光谱 (x, y) is the image pixel value after spectral equalization, β is the detail enhancement coefficient, which controls the influence weight of high-resolution details in the enhanced image, ΔI 高分辨率 (x, y) is the high-frequency component of the high-resolution image;

[0117] For ΔI 高分辨率 (x, y) is the high-frequency component of the high-resolution image calculated according to the following formula:

[0118] ΔI 高分辨率 (x,y)=I 高分辨率 (x,y)-Gaussian blur (I 高分辨率 (x,y));

[0119] In the formula, I 高分辨率 (x, y) is the original pixel value of the high-resolution image at pixel (x, y), and Gaussian blur (I 高分辨率 (x,y)) is the high-resolution image I 高分辨率 The low-frequency part obtained after Gaussian blurring of (x, y) is obtained by processing the image using a Gaussian blurring algorithm to remove high-frequency details and retain the low-frequency component of the image. The low-frequency component corresponds to an area where the brightness of the object in the image is uniform or changes slowly, which is well known to those skilled in the art and will not be described in detail in this embodiment.

[0120] S502, global equalization: for the adjusted image I 增强 Perform global histogram equalization to improve the dynamic range and visual effect of the image.

[0121] S503, generate the final fused image: output the adjusted image I 最终融合 , containing balanced spectral and detail information.

[0122] The fusion optimization unit evaluates the quality of the preliminary fusion image, adjusts and optimizes the image according to the set evaluation index, and thus generates a high-quality final fusion image; the fusion optimization unit quantifies the evaluation index (including the spectral information fidelity Q 光谱 and detail information retention Q 细节 ), judge whether the spectral information fidelity, detail information retention and overall quality of the preliminary fusion image meet the system requirements. Specifically, the spectral information fidelity Q 光谱 Calculated according to the following formula:

[0123] In the formula, I 多光谱 (x, y) is the pixel value of the original multispectral image, and its value can be directly obtained from the image data through the pixel value of the multispectral image. 融合 (x,y) is the pixel value of the initial fusion image, is the global mean of the multispectral image. Calculate according to the following formula:

[0124]

[0125] Where M is the image width (number of columns) and N is the image height (number of rows).

[0126] Detail information retention Q细节 Calculate according to the following formula:

[0127]

[0128] In the formula, is the gradient vector of the fused image, is the gradient vector of the high-resolution image; where, for the gradient of the fused image Calculated according to the following formula:

[0129]

[0130] In the formula, G x 融合 is the gradient of the fused image in the horizontal direction (column direction), G y 融合 is the gradient of the fused image in the vertical direction (row direction), is the gradient amplitude of the fused image;

[0131] Gradients for high-resolution images Calculated according to the following formula:

[0132]

[0133] In the formula, G x 高分辨率 is the gradient of the high-resolution image in the horizontal direction (column direction), G y 高分辨率 is the gradient of the high-resolution image in the vertical direction (row direction), is the magnitude of the gradient of the high-resolution image;

[0134] Overall quality index Q 总 Calculate according to the following formula: Q 总 =η·Q 光谱 +λ·Q 细节 ;

[0135] Where η and λ are weight coefficients, which usually satisfy η+λ=1. When the application scenario focuses more on spectral fidelity (such as crop classification, vegetation monitoring, etc.), η should be larger, and when the application scenario focuses more on spatial details (such as building boundary detection, road monitoring, etc.), λ should be larger. When it is necessary to take into account both spectral and detailed information (such as disaster monitoring, mixed ground feature change analysis), η≈λ.

[0136] In addition, in this embodiment, an example of the value of the weight coefficient is also provided. Specifically: 1) in the scenario of agriculture and vegetation monitoring (paying more attention to spectral information in order to distinguish different crops or vegetation types), η = 0.7, λ

[0137] =0.3; 2) In the scenario of urban building and infrastructure monitoring (paying more attention to spatial details in order to accurately detect building boundaries or road distribution), then η = 0.3, λ = 0.7; 3) In the scenario of mixed scenarios (disaster monitoring, land use analysis) (it is necessary to integrate spectral and detail information to ensure the comprehensiveness of change analysis), then η = 0.5, λ = 0.5; 4) In the scenario of water body and pollution monitoring (paying attention to the change of spectral characteristics (such as the change of reflectivity with the degree of pollution)), then η = 0.8, λ = 0.2;

[0138] If Q 总 If the set threshold is not reached, the optimization process is entered, otherwise, the fusion process is performed with the current spectral consistency weight and detail richness weight;

[0139] The optimization steps include: S600, weight redistribution: dynamically adjust the spectral weight W 光谱 and detail weight W 细节 , where the new spectral weight W' 光谱 and the new detail weight W' 细节 Determined according to the following formula:

[0140]

[0141] Where W 光谱 is the original spectral weight, W 细节 is the original detail weight, γ, δ are optimization factors, which control the intensity of weight adjustment, and the value range is: γ, δ∈[0,2]. In this embodiment, an example of the value of the optimization factor is provided, specifically: 1) vegetation monitoring (spectral information priority), the quality of high spectral information is crucial to vegetation classification, and the spectral weight needs to be improved first, then γ=0.8, δ=0.3. 2) building boundary monitoring (detail information priority), higher resolution and geometric details are required, and the detail weight is improved first, then γ=0.3, δ=0.9. 3) disaster monitoring (comprehensive spectrum and detail information), it is necessary to balance the spectrum and detail information to analyze the changes in the mixed area, then γ=0.5, δ=1.5. 4) water pollution monitoring (spectral information is extremely important), high spectral accuracy is crucial to detecting the degree of pollution and water composition, then γ=1.0, δ=0.2.

[0142] S601, re-evaluate the quality of the optimized image. If the target is not reached, repeat the above steps.

[0143] The fusion optimization unit evaluates the following two key indicators of the preliminary fusion image (including the information index H 融合 and edge density D 边缘 ), determine whether the grid size needs to be adjusted:

[0144] Where L is the grayscale level. For an 8-bit grayscale image, L = 256 (the grayscale value range is 0 to 255), P i is the probability of the i-th gray value, and its value satisfies: n i is the number of pixels with gray value i, N is the total number of pixels in the image, and the calculation formula is: N=M×N, M is the image width (number of pixels), and N is the image height (number of pixels).

[0145] The specific calculation process is as follows: 1) Calculate the grayscale histogram: Count the grayscale values ​​of all pixels in the image to generate a grayscale histogram. The number of pixels n at each grayscale level i i is the number of gray levels in the histogram. 2) Calculate the gray probability P i ; 3) Calculate the information index H 融合 .

[0146] If H 融合 or D 边缘 If the threshold range set by the system is not met (such as too low information or insufficient edge information), the grid size adjustment is triggered; specifically, H 融合 and D 边缘 None of them meet the threshold range set by the system, that is, H 融合 Does not fall within the information volume monitoring range [TH2, TH2] set by the system, and D 边缘 If it does not fall into the edge density monitoring range [TD2, TD2] set by the system, a large adjustment of the grid is triggered; H 融合 or D 边缘 None of them meet the threshold range set by the system, that is, H 融合 Does not fall within the information volume monitoring range [TH2, TH2] set by the system, or D 边缘 If it does not fall into the edge density monitoring range [TD2, TD2] set by the system, a minor adjustment of the grid is triggered;

[0147] Among them, the information volume monitoring range set by the system is [TH2, TH2], the edge density monitoring range set by the system is

[0148] [TD2, TD2] are set by the system according to the actual situation. This is a technical means well known to those skilled in the art, so it will not be described in detail in this embodiment. 融合 or D 边缘 When none of the grids meet the threshold range set by the system, the steps of adjusting the grid size (steps S700 to S704) include:

[0149] S700, calculating a new grid size, that is, taking a larger grid reduction to improve the resolution to capture more details and information: The grid size is increased by a larger amount according to the following formula:

[0150] s 新 =s 当前 (1+α);

[0151] In the above formula, s 当前 is the size of the current grid cell, s 新 is the adjusted grid unit size, α is the adjustment scale factor, usually [0.1, 0.5].

[0152] S701. Re-dividing the grid: Re-dividing the multispectral image and the high-resolution image according to the new grid size.

[0153] The boundary range of each grid cell is updated as: G ij =[i·s 新 ,(i+1)·s 新 )×[j·s 新 ,(j+1)·s 新 );

[0154] S702, recalculating grid-level weights: recalculating spectral consistency weights and detail richness weights according to the new grid division;

[0155] S703, re-fusion and optimization: based on the new grid division and weights, re-execute pixel-level weighted fusion and subsequent optimization.

[0156] S704, Dynamic adjustment and iteration: Continuously monitor the quality index H of the fused image 融合 and D 边缘 .

[0157] If the adjusted raster size still does not meet the quality requirements, the adjustment process is repeated until the system threshold is met or the maximum number of adjustments is reached.

[0158] Through the cooperation of the data acquisition module and the image fusion module, efficient fusion of multi-source data (satellite images and drone images) is achieved, so that the output image has both the fidelity of spectral information and high-resolution spatial details, ensuring that the multi-source remote sensing data still maintains information integrity and quality after fusion.

[0159] If H 融合 or D 边缘 All of them meet the threshold range set by the system, that is, H 融合 falls within the information volume monitoring range [TH2, TH2] set by the system, and D 边缘 If it falls within the edge density monitoring range [TD2, TD2] set by the system, the grid size remains unchanged. 融合 or D 边缘 One of them does not meet the system set threshold range, that is, H融合 Does not fall within the information volume monitoring range set by the system [TH2, TH2] but D 边缘 falls within the edge density monitoring range [TD2, TD2] set by the system, or H 融合 Falling within the information volume monitoring range set by the system [TH2, TH2] but D 边缘 If it does not fall into the edge density monitoring range [TD2, TD2] set by the system, then steps S800 to S804 are executed:

[0160] S800, calculating a new grid size, and performing a small grid adjustment;

[0161] If H 融合 Not satisfied (insufficient information) (i.e., H 融合 Not falling within the information volume monitoring range set by the system [TH2, TH2]): Slightly reduce the grid size and increase the amount of information.

[0162] If D 边缘 Not satisfied (insufficient details) (i.e., H 融合 Falling within the information volume monitoring range set by the system [TH2, TH2]): slightly reduce the grid size and enhance the edge information.

[0163] Among them, the small reduction in grid size is determined by the following formula: 新 =s 当前 ·(1-β); where s 当前 is the size of the current grid cell, s 新 is the adjusted grid cell size, β is the adjustment scale factor, usually [0.1, 0.5].

[0164] S801, re-dividing the grid: re-dividing the multispectral image and the high-resolution image according to the new grid size. The boundary range of each grid cell is updated to:

[0165] G ij =[i·s 新 ,(i+1)·s 新 )×[j·s 新 ,(j+1)·s 新 );

[0166] S802, recalculating grid-level weights: recalculating spectral consistency weights and detail richness weights according to the new grid division;

[0167] S803, re-fusion and optimization: based on the new grid division and weights, re-execute pixel-level weighted fusion and subsequent optimization.

[0168] S804. Dynamic adjustment and iteration: Continuously monitor the quality index H of the fused image融合 and D 边缘 .

[0169] If the adjusted raster size still does not meet the quality requirements, the adjustment process is repeated until the system threshold is met or the maximum number of adjustments is reached.

[0170] In this embodiment, a value range of the adjusted proportional coefficients α, β is provided: [0.1, 0.5], that is, an adjustment ratio of 10% to 50%; for the value trend of the adjustment proportional coefficient α: small scene (fine-grained adjustment): when it is necessary to adjust the invalid grids in a small area, the α value is small (such as 0.1~0.2) to finely control the increase in the grid; large scene (coarse-grained adjustment): when it is necessary to quickly reduce the number of invalid grids, the α value is large (such as 0.3~0.5), which significantly improves the coverage of the grid.

[0171] For the value trend of adjusting the scale factor β: small scene (refined grid): when higher resolution is needed to capture details, the β value is smaller (such as 0.1~0.2), which slightly reduces the grid; large scene (fast refinement): when the grid resolution needs to be increased quickly, the β value is larger (such as 0.3~0.5), which significantly reduces the grid size.

[0172] At the same time, a specific value example is provided: 1) In the scenario of vegetation monitoring (high precision requirement), the spatial variation of vegetation distribution is accurately captured, and the grid unit size is reduced to improve the resolution, then α = 0.1, β = 0.4; 2) In the scenario of disaster monitoring (high coverage requirement), invalid grids are quickly reduced, the coverage of effective data is improved, and the grid unit size is increased, then α = 0.5, β = 0.2; 3) In the scenario of urban building boundary monitoring (balanced requirement), it is necessary to capture the details of the building boundary and a certain coverage range, then α = 0.2, β = 0.2; 4) In the scenario of water pollution monitoring (dynamic adjustment), when the water range changes greatly, the grid size is dynamically adjusted to adapt to different areas, then α = 0.3, β = 0.3. In short, the values ​​of the adjustment ratio coefficients α and β need to be combined with the specific usage scenario and input from the human-computer interaction interface. This is a technical means well known to technicians in this field, so in this embodiment, it will not be repeated one by one.

[0173] Optionally, the feature extraction module includes a spectral feature extraction unit, a texture feature extraction unit, a shape feature extraction unit and a multidimensional feature encoding unit. The spectral feature extraction unit analyzes the spectral information of the fused image, the texture feature extraction unit uses the image grayscale change information to analyze the texture characteristics of the ground object (ground object), the shape feature extraction unit analyzes the shape characteristics of the ground object in the fused image based on the edge and geometric structure, and the multidimensional feature encoding unit integrates the extracted spectral, texture and shape features into a unified multidimensional feature description matrix F(x,y), and standardizes them to form a standardized multidimensional feature description matrix F'(x,y).

[0174] The spectral feature extraction unit extracts the reflectance value of each band (red light, green light, near infrared):

[0175] First, the multi-band data of the fused image I k (x,y): k = 1, 2, ..., K, indicating the number of bands in the image; each band is a grayscale image that records the reflectance value of the pixel corresponding to the band. The spectral characteristics of each pixel are expressed as: S(x,y) = [B1(x,y), B2(x,y), ..., B k (x,y)], where B k (x, y) is the value of the kth band. For all pixels (x, y), the pixel value of the corresponding position is read from each band image. The texture feature extraction unit is based on the statistical characteristics of the gray level co-occurrence matrix (GLCM), that is, the gray level co-occurrence matrix is ​​calculated: P(i, j, d, θ): represents the frequency of occurrence P(i, j) of pixel pairs with gray values ​​​​i and j at distance d and direction θ. In this embodiment, the specific directions are: 0°, 45°, 90°, 135°; the distance d is 1 pixel or 2 pixels. Among them, the frequency of occurrence P(i, j) of pixel pairs with gray values ​​​​i and j is calculated according to the following formula:

[0176]

[0177] Among them, the number of occurrences is through the grayscale image I(x,y), the size is H×W, the pixel grayscale value range is [0,L-1], where L is the grayscale level. Specified direction θ and distance d: Direction θ: commonly used 0°, 45°, 90°, 135°; Distance d: usually 1 or 2 pixels, and count the number of pixel pairs that meet the (i, j) condition. Next, traverse each pixel (x, y): check whether its neighboring pixel (x′, y′) is within the image range; if it is within the range, check (I(x, y)=i)∧(I(x′, y′)=j); if the condition is met, increase the count C(i, j)=C(i, j)+1, and output the count matrix C(i, j);

[0178] In this embodiment, the output count matrix C(i, j) is set to the number of occurrences; the determination of the total number of gray levels includes counting the number of pixel pairs that meet the conditions by traversing the image:

[0179] Iterate over each pixel (x,y) in the grayscale image.

[0180] Determine whether a pixel (x, y) has a valid neighboring pixel (x′, y′): The neighboring pixel must be within the image range; if it is within the range, the pixel pair is considered valid. Accumulate the number of all valid pixel pairs, and if it is within the range, increase the count N pairs =N pairs +1, output N pairs In this embodiment, N pairs Set to the total grayscale logarithm;

[0181] According to the gray level co-occurrence matrix GLSM, the texture index of the image grayscale is determined. The texture index includes contrast, entropy, and homogeneity. The specific index is calculated according to the following formula:

[0182]

[0183] Where i, j are the indices of grayscale values, i, j∈[0,L-1], where L is the number of grayscale levels (usually 256);

[0184] The shape feature extraction unit analyzes the connected area in the edge map based on the binarization result or edge detection result of the fused image, extracts the contour information of the object, calculates the total number of pixels (area) in the connected area of ​​the fused image, and the total number of boundary pixels (perimeter), thereby obtaining the shape factor ShapeFactor:

[0185]

[0186] Wherein, area is the area, and perimeter is the perimeter; the multidimensional feature encoding unit combines the spectrum, texture, and shape features of each pixel or region into a feature vector: F(x,y) = [S(x,y), T(x,y), C(x,y)]; in this embodiment, S(x,y) = [B1(x,y), B2(x,y), …, B k (x,y)], where B k (x, y) is the value of the kth band, T(x, y) = [Contrast, Entropy, Homogeneity], C(x, y) = [area, perimeter, ShapeFactor].

[0187] The multidimensional feature encoding unit normalizes the feature vector F(x,y) to obtain F'(x,y):

[0188]

[0189] In the formula, F min is the minimum value of all features in the feature vector F(x,y), min =min(S min , T min , C min ), S min is the global minimum of the spectral feature, T min is the global minimum value of texture features, C min is the global minimum value of the shape feature, F max is the maximum value of all features in the feature vector F(x,y), max =max(S max , T max , C max ), S max The global maximum of the spectral feature, T max is the global maximum value of texture features, C max is the global maximum value of the shape feature.

[0190] For the global minimum value S of the spectral feature min and the global maximum S of the spectral features max , determined according to the following formula: S min =min(B k，min ), S max =max(B k，max ); Specifically, traverse all bands k=1,2,…,K, and count the minimum value B of each band k,min and the maximum value B k,max .

[0191] For the global minimum value T of texture features min and the global maximum value T of texture features max Determine according to the following formula: T min =min(Contrast min , Entropy min , Homogeneity min );T max =max(Contrast max , Entropy max , Homogeneity max ) ; that is, traverse the texture features of each pixel and count the minimum and maximum values ​​of each dimension. For the global minimum value C of the shape feature minand the global maximum value C of the shape feature max , determined according to the following formula: C min =min(area min , perimeter min , ShapeFactor min );C max =max(area max , perimeter max , ShapeFactor max ), that is, traverse the shape features of the object and directly count the minimum value C min and the maximum value C max .

[0192] Through the cooperation of the image fusion module and the feature extraction module, efficient feature analysis of the fused image is achieved, and the multi-dimensional feature description matrix F(x,y) is extracted, making the anomaly detection of the evaluation module more accurate and ensuring the smoothness and reliability of the data processing chain.

[0193] Optionally, the evaluation module obtains the multi-dimensional feature description matrix F'(x, y) and calculates the comprehensive abnormal index Abnormal(x, y) of the pixel point (x, y) according to the following formula:

[0194] Wherein, d is the feature dimension of the standardized multi-dimensional feature description matrix F'(x, y). In this embodiment, d=K+d T +d C , that is, the total dimension of spectral features, texture features, and shape features, Z i (x, y) is the standardized anomaly score of the pixel (x, y) in the i-th feature dimension, CV i is the coefficient of variation of the i-th feature dimension, α is the adjustment coefficient, and its value is set by the system according to the actual situation;

[0195] The standardized anomaly score Z of the pixel (x, y) in the i-th feature dimension i (x, y), calculated according to the following formula:

[0196]

[0197] In the formula, F i ′(x,y) is the value of the i-th feature in the multidimensional feature description matrix, μ i is the global mean of the i-th feature dimension:

[0198] Where W is the image width, H is the image height, and F i ′(x,y) is the value of the i-th feature in the multidimensional feature description matrix.

[0199] The global standard deviation σ of the i-th feature dimension i Calculated according to the following formula:

[0200]

[0201] Where W is the image width, H is the image height, and F i ′(x,y) is the value of the i-th feature in the multidimensional feature description matrix.

[0202] The coefficient of variation CV of the i-th feature dimension i Calculated according to the following formula:

[0203]

[0204] In the formula, σ i is the global standard deviation of the i-th feature dimension, μ i is the global mean of the i-th feature dimension.

[0205] In this embodiment, the value range of the adjustment coefficient α is: α≥0, and a specific value example is provided: 1) For the scene of vegetation anomaly detection in remote sensing images (detecting the health status of vegetation in agricultural areas, discovering anomalies through spectral characteristics (such as NDVI index), and texture characteristics may be affected by terrain changes, but the significance for vegetation health is relatively small), then α=0.5; 2) In the scene of building shape anomaly detection in urban planning (in urban remote sensing images, shape characteristics (such as aspect ratio, shape factor) are used to detect building structure anomalies (such as too slender or irregular shapes), and spectral characteristics and texture characteristics are only used as auxiliary information), then α=2.0; 3) In the scene of water area change monitoring (detecting changes in water area (such as lakes, rivers, etc.), spectral characteristics are the key, and shape characteristics and texture characteristics are secondary), then α=0.8; In short, the specific value of the adjustment coefficient α needs to be set according to the actual scene and input from the human-computer interaction interface. This is a technical means well known to technicians in this field, so in this embodiment, it will not be repeated one by one.

[0206] If the comprehensive anomaly index Abnormal(x, y) of the pixel point (x, y) does not fall within the evaluation threshold range [B1, B2] set by the system, it indicates that the pixel point (x, y) is abnormal. If the comprehensive anomaly index Abnormal(x, y) of the pixel point (x, y) falls within the evaluation threshold range [B1, B2] set by the system, it indicates that the pixel point (x, y) is not abnormal. The evaluation threshold range [B1, B2] set by the system is set by the system or the administrator according to the actual scenario and input through the human-computer interaction interface. This is a well-known technical means in the art, and those skilled in the art can query relevant technical manuals to obtain this technology. Therefore, it will not be elaborated one by one in this embodiment. Through the cooperation of the feature extraction module and the evaluation module, the efficient extraction and multi-dimensional comprehensive analysis of spectral features, texture features, and shape features are realized, enabling the system to evaluate the changes and anomalies in the monitoring area from multiple angles, ensuring the accuracy and comprehensiveness of anomaly detection, and enhancing the adaptability to complex scenarios.

[0207] The result output module includes an interaction unit and an early warning unit. The interaction unit obtains the evaluation result and interacts with the early warning unit. The early warning unit gives an early warning prompt of the evaluation result to the administrator. The interaction unit includes a data interface and an interaction logic processor. The data interface obtains the comprehensive anomaly index Abnormal(x, y) of the evaluation module. The interaction logic processor analyzes the relationship between the comprehensive anomaly index and the early warning threshold range [B1, B2], marks the abnormal points or areas with different levels, and transmits them to the early warning unit. Among them, the interaction logic processor makes a threshold comparison: if Abnormal(x, y) < B1, it is marked as a low-value anomaly; if Abnormal(x, y) > B2, it is marked as a high-value anomaly; if B1 ≤ Abnormal(x, y)

[0208] ≤ B2, it is marked as normal.

[0209] The early warning unit includes an early warning rule library, an early warning generator, and a notification sender. The early warning rule library stores early warning rules corresponding to the anomaly levels (such as color identification, information level, notification method); the early warning generator generates early warning prompt information based on the data transmitted by the interaction unit, including graphical display and text description; the notification sender sends the early warning information to the administrator through the set channels (such as text messages, emails, system notifications). For example: anomaly: red identification, send a text message notification; normal: green identification, only system log records. During the early warning notification process, in the visual interface, the abnormal area is highlighted: combined with the anomaly index matrix, a heat map is displayed in the image; color coding (red, green, etc.) is used to intuitively identify abnormal areas at different levels.

[0210] In addition, the present invention also provides a remote sensing image processing method, which includes the following steps: S1, dynamically acquiring satellite images of the monitored area and image data acquired by a drone through a data acquisition module;

[0211] S2, pre-processing the satellite image and image data through the image fusion module to generate a multispectral image and a high-resolution image respectively, and fusing the multispectral image and the high-resolution image to form a fused image;

[0212] S3, extracting spectral features, texture features and shape features from the fused image through a feature extraction module to generate a multi-dimensional feature description matrix;

[0213] S4, evaluating the abnormal position according to the multi-dimensional feature description data through the evaluation module to form an evaluation result;

[0214] S5. Generate early warning prompts according to the evaluation results through the result output module.

[0215] Optionally, the remote sensing image processing method further includes: in step S3, dividing the multispectral image and the high-resolution image generated after preprocessing through an image fusion module to generate grid units of a fixed size.

[0216] Optionally, the remote sensing image processing method also includes: calculating the spectral consistency weight of the multispectral image in each grid cell through the analysis unit, and evaluating the spatial detail richness of the high-resolution image in each grid cell, and performing fusion processing based on the spectral consistency weight and the detail richness weight.

[0217] Through the cooperation of the evaluation module and the result output module, real-time detection and classification of abnormal locations are achieved. Through the linkage between the interactive unit and the early warning unit, timely early warning prompts are provided to managers, ensuring the system's response speed to potential risks and ease of operation.

[0218] Through the mutual cooperation of data acquisition module, feature extraction module, image fusion module, evaluation module and result output module, the remote sensing image processing system has high efficiency, accuracy and operability in multi-source data fusion, anomaly detection and result output, ensuring that the entire system can conduct efficient analysis and early warning of the monitored area in real time and comprehensively.

[0219] Embodiment 2: This embodiment should be understood to include all the features of any of the above embodiments, and further improve upon them. Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6As shown, the remote sensing image processing system also includes a change monitoring module, which obtains a multidimensional feature description matrix F′(x,y) standardized by a sampling period, calculates a change index Change(x,y), and triggers a change warning prompt; the change monitoring module includes a change analysis unit and a report generation unit, the change analysis unit analyzes the multidimensional feature description matrix F′(x,y) standardized by a sampling period to form an analysis result, and the report generation unit prompts the manager according to the analysis result.

[0220] The change analysis unit obtains a multidimensional feature description matrix F′(x, y) standardized by a sampling period and calculates the change rate R of the i-th element according to the following formula: i (x, y):

[0221]

[0222] In the formula, F′ i (x, y, t) is the i-th element value of the multidimensional feature description matrix standardized in the current cycle, F′ i (x, y, t-1) is the i-th element value of the multidimensional feature description matrix standardized in the previous period, ∈ is a small constant value, usually set to a very small positive number, for example ∈ = 10 -6 ;

[0223] The change analysis unit obtains the change rate R i (x, y) calculates the change index Change(x, y):

[0224]

[0225] In the formula, d is the feature dimension of the standardized multidimensional feature description matrix F'(x,y), R i (x, y) is the rate of change of the i-th element.

[0226] If the change index Change(x, y) falls within the change warning threshold range [T low , T high ], indicating that the change is within the acceptable range and no alarm is needed. If the change index Change(x, y) does not fall within the change warning threshold range [T low , T high ], indicating that the change is abnormal and an alarm needs to be triggered. Specifically, if Change(x,y)<T low , it is in a low value abnormal state, indicating that the change is too small, which may reflect inaccurate data or small changes; if Change(x,y)>T high , it is in a high value abnormal state, indicating that the change is too large, which may reflect that there are drastic changes or risks in the monitored object.

[0227] The report generation unit includes a data caller and a report generator. The data caller calls the analysis results of the change analysis unit (change index Change (x, y) and the change warning threshold range [T low , T high ]), the report generator generates a corresponding text description according to the comparison result of the Su data caller and prompts the manager;

[0228] In this embodiment, an example of text description is as follows: Figure 5 At the same time, through the cooperation between the change monitoring module and the feature extraction module, the system can effectively realize the comprehensive analysis and abnormal warning of the target area based on the accurate extraction of multi-dimensional features and real-time monitoring of dynamic changes, ensuring that the whole system has the advantages of high precision, high real-time performance and intelligence.

[0229] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A remote sensing image processing system, comprising an unmanned aerial vehicle, characterized in that: The remote sensing image processing system also includes a data acquisition module, a feature extraction module, an image fusion module, an evaluation module and a result output module; The data acquisition module dynamically acquires satellite images of the monitored area and image data acquired by the drone, the image fusion module pre-processes the satellite images and image data to generate a multispectral image and a high-resolution image respectively, and fuses the multispectral image and the high-resolution image to form a fused image, the feature extraction module extracts spectral features, texture features and shape features from the fused image to generate a matrix containing multidimensional feature descriptions, the evaluation module evaluates the abnormal position according to the multidimensional feature description data to form an evaluation result, and the result output module generates an early warning prompt according to the evaluation result; The image fusion module includes a preprocessing unit, a grid division unit, an analysis unit, a weighted fusion unit and a fusion optimization unit. The preprocessing unit preprocesses the satellite image and image data to generate a multispectral image and a high-resolution image respectively. The grid division unit divides the input multispectral image and high-resolution image into grid units of fixed size. The analysis unit calculates the spectral consistency weight of the multispectral image in each grid unit and evaluates the spatial detail richness of the high-resolution image in each grid unit. The weighted fusion unit performs fusion processing based on the spectral consistency weight and the detail richness weight. The fusion optimization unit performs quality evaluation and optimization on the preliminary fused image to form a fused image. The fusion optimization unit evaluates the information volume and quantitative evaluation index of the preliminary fusion image, and if the index does not meet the monitoring threshold of the system, the size and weight of the grid unit are readjusted.

2. A remote sensing image processing system according to claim 1, characterized in that: The data acquisition module includes an image acquisition unit and an image acquisition unit. The image acquisition unit retrieves satellite image data of the monitored area, and the image acquisition unit collects image data collected by a drone. The image acquisition unit is loaded on the drone and follows the drone to collect image data of the monitored area.

3. A remote sensing image processing system according to claim 2, characterized in that: The grid division unit includes a spatial alignment component, a grid generation component, a multi-source mapping grid mapping component and a grid quality control component. The spatial alignment component aligns the spatial coordinate systems of the multispectral image and the high-resolution image. The grid generation component generates a fixed-size grid unit according to the spatial range and resolution of the image. The multi-source mapping grid mapping component maps the pixel values ​​of the multispectral image and the high-resolution image to the corresponding grid units. The grid quality control component evaluates the quality evaluation result of the grid division and adjusts the grid size or range according to the evaluation result.

4. A remote sensing image processing system according to claim 3, characterized in that: The analysis unit includes a spectral consistency component, a spatial detail evaluation component and a weight normalization component. The spectral consistency component analyzes the spectral distribution characteristics of the multispectral image in each grid unit and calculates the spectral consistency weight. The spatial detail evaluation component evaluates the detail richness of the high-resolution image in each grid unit. The weight normalization component normalizes the spectral consistency weight and the detail richness weight.

5. A remote sensing image processing system according to claim 4, characterized in that: The weighted fusion unit includes a weight mapper, a pixel fusion unit, a boundary smoothing component and a global adjustment component. The weight mapper receives the spectral consistency weight and detail richness weight calculated by the analysis unit and maps them to the pixel level of each grid unit. The pixel fusion unit performs weighted superposition on the pixels in each grid unit to generate a preliminary fused image. The boundary smoothing component smoothes the boundaries between grid units. The global adjustment component performs global adjustment on the fused preliminary image.

6. A remote sensing image processing system according to claim 5, characterized in that: The feature extraction module includes a spectral feature extraction unit, a texture feature extraction unit, a shape feature extraction unit and a multidimensional feature encoding unit. The spectral feature extraction unit analyzes the spectral information of the fused image, the texture feature extraction unit uses the image grayscale change information to analyze the texture characteristics of the object, the shape feature extraction unit analyzes the shape characteristics of the object in the fused image based on the edge and geometric structure, and the multidimensional feature encoding unit integrates the extracted spectral, texture and shape features into a unified multidimensional feature description matrix F(x,y), and standardizes them to form a standardized multidimensional feature description matrix F'(x,y).

7. A remote sensing image processing system according to claim 6, characterized in that: The evaluation module describes the multidimensional feature matrix F'(x, y) and calculates the comprehensive abnormal index Abnormal(x, y) of the pixel point (x, y) according to the following formula: Where d is the feature dimension of the standardized multidimensional feature description matrix F'(x,y), Z i (x, y) is the standardized anomaly score of the pixel (x, y) in the i-th feature dimension, CV i is the coefficient of variation of the i-th feature dimension, α is the adjustment coefficient, and its value is set by the system according to the actual situation; If the comprehensive abnormal index Abnormal(x, y) of the pixel point (x, y) does not fall within the evaluation threshold range [B1, B2] set by the system, it means that the pixel point (x, y) is abnormal.

8. A remote sensing image processing method, applied to a remote sensing image processing system as claimed in claim 7, characterized in that: The remote sensing image processing method comprises the following steps: S1. Dynamically acquire satellite images of the monitored area and image data collected by the drone through a data acquisition module; S2, pre-processing the satellite image and image data through the image fusion module to generate a multispectral image and a high-resolution image respectively, and fusing the multispectral image and the high-resolution image to form a fused image; S3, extracting spectral features, texture features and shape features from the fused image through a feature extraction module to generate a multi-dimensional feature description matrix; S4, evaluating the abnormal position according to the multi-dimensional feature description data through the evaluation module to form an evaluation result; S5. Generate early warning prompts according to the evaluation results through the result output module.

9. A remote sensing image processing method according to claim 8, characterized in that: The remote sensing image processing method further includes: in step S3, dividing the multispectral image and the high-resolution image generated after preprocessing by an image fusion module to generate grid units of fixed size.

10. A remote sensing image processing method according to claim 9, characterized in that: The remote sensing image processing method also includes: calculating the spectral consistency weight of the multispectral image in each grid unit through an analysis unit, evaluating the spatial detail richness of the high-resolution image in each grid unit, and performing fusion processing based on the spectral consistency weight and the detail richness weight.

Images