A human activity change detection method based on multi-information fusion image

Abstract

This invention provides a method for detecting changes in human activity based on multi-information fusion images, belonging to the field of Earth monitoring technology. The method involves preprocessing raw images of high-intensity seismic activity areas to obtain synthetic aperture radar (SAR) images of the study area; performing polarimetric synthesis and polarimetric decomposition on the SAR images of the study area to obtain polarization information images; extracting intensity information from the SAR images of the study area to obtain intensity information change images; obtaining polarization fusion feature information images through image fusion and comparative detection based on the raw images of the high-intensity seismic activity areas and the polarization information images; fusing the intensity information change images with the polarization fusion feature information images and the raw images of the high-intensity seismic activity areas respectively; and performing detection and comparative analysis on the results to obtain the human activity change detection results. This invention solves the problem of insufficient timeliness caused by existing change detection methods that rely solely on optical images.

Description

Technical Field

[0001] This specification relates to the field of Earth monitoring technology, and in particular to a method for detecting changes in human activity based on multi-information fusion images. Background Technology

[0002] While optical remote sensing imagery can directly interpret changes in buildings in high-intensity seismic intensity areas, it is susceptible to cloud and fog effects, suffers from low timeliness, is limited by weather conditions, and is insensitive to ground roughness. This limits its ability to comprehensively detect changes in buildings in high-intensity seismic intensity areas. In contrast, Synthetic Aperture Radar (SAR) satellites can acquire surface structure characteristics of ground features, have short revisit cycles, and are unaffected by cloud and fog. They can rapidly detect changes in ground features using multiple imagery periods, compensating for the shortcomings of optical remote sensing. Therefore, it is crucial to fully utilize the characteristic information of SAR imagery for research on detecting changes in buildings in high-intensity seismic intensity areas. Furthermore, changes in buildings are a core manifestation of changes in human activity; detecting changes in buildings can reflect changes in human activity.

[0003] Optical satellites excel at providing high-resolution and color images, enabling them to identify minute changes and features on the Earth's surface, and offer high accuracy in target recognition and classification. However, these satellites are limited to daytime observations, and the images captured are often obscured by clouds and fog, making it difficult to receive surface radiation information. Currently, accurate de-clouding and de-fogging of remote sensing images is challenging, and information about ground features obscured by clouds and fog is difficult to obtain, hindering the identification and classification of ground features in remote sensing images, as well as the detection of building changes in high-intensity seismic zones. Against this backdrop, radar satellites are widely used due to their ability to penetrate clouds and fog, and they can also penetrate vegetation to acquire more surface features. However, most current research on extracting ground feature features from SAR images focuses on the identification and detection of different types of ground features, with limited research on detecting changes in ground features at the same location over different times. Therefore, how to accurately detect ground feature changes using SAR imagery is a current research focus. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a human activity change detection method based on multi-information fusion images, which solves the problem of insufficient timeliness caused by existing change detection methods relying solely on optical images.

[0005] To achieve the aforementioned objectives, the present invention employs the following technical solution: a method for detecting changes in human activity based on multi-information fusion images, comprising: S1: Preprocess the original images of high-intensity areas containing human activities to obtain synthetic aperture radar images of the study area; S2: Perform polarization synthesis and polarization decomposition on the synthetic aperture radar image of the study area to obtain a polarization information image; S3: Extract intensity information from the synthetic aperture radar image of the study area to obtain an image showing the change in intensity information; S4: Based on the original images of the high-intensity region and the polarization information image, a polarization fusion feature information image is obtained through image fusion and comparison detection; S5: The intensity information change image is fused with the polarization fusion feature information image and the original image of the high-intensity area, respectively. The first fusion result and the second fusion result are detected and compared to obtain the human activity change detection result, thus completing the human activity change detection based on multi-information fusion image.

[0006] Further, S1 includes: Multi-view processing, filtering, geocoding, and radiometric calibration were performed on the original Gaofen-3 synthetic aperture radar images. The synthetic aperture radar images of the study area were obtained by cropping. Among them, the original Gaofen-3 synthetic aperture radar images belong to the original images of high-intensity areas containing human activities.

[0007] Further, S3 includes: Based on image intensity information, the synthetic aperture radar images of the study area were divided into K classes, resulting in multiple cluster centers. The membership degrees of the synthetic aperture radar image of the study area and the cluster center are calculated, and the intensity information of the synthetic aperture radar image of the study area is extracted to obtain an intensity information change image.

[0008] Furthermore, the expression for the membership degree is: ; ; in, Represents sample points With cluster center membership degree Representing data points i To other cluster centers k distance, Represents sample points With cluster center distance, i Indicates the index of the data point. j This represents the index of the currently calculated cluster center, and c represents the indices of all cluster centers traversed during the summation process. k This represents the total number of clusters. m Represents the fuzzy index. Indicates the cluster center. Represents sample points, Indicates membership degree ofm The power is used to weight the contribution of data points to cluster centers, where n represents the total number of sample points.

[0009] Further, S4 includes: S410: Based on the original image of the high-intensity area and the polarization information image, an original fused image is obtained through image fusion; S420: The original fused image is analyzed using an image segmentation convolutional neural network to obtain a polarization fusion feature information image; wherein, the image segmentation convolutional neural network is obtained through training.

[0010] Further, S410 includes: Based on the original image of the high-intensity area and the polarization information image, the resampled pixel value is obtained by resampling using the weighted average of the nearest neighbor pixels; The resampled pixel values ​​are interpolated to obtain horizontal and vertical interpolation values; Based on the resampled pixel values, the horizontal interpolation, and the vertical interpolation, geographic coordinates are obtained by constructing a coordinate linear transformation relationship; The geographic coordinates are stacked by band dimensions, and multiple single-band images are fused into a multi-band dataset to obtain the original fused image.

[0011] Furthermore, the expression for the horizontal interpolation is: ; ; in, Indicated on the y-axis The horizontal axis on the horizontal line x The interpolation result at that point, Indicated on the y-axis The horizontal axis on the horizontal line x The interpolation result at that point, Represents the x-coordinate of the target point. , Represents the x-coordinate of a known data point. , Represents the y-coordinate of a known data point. The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value; The expression for the vertical interpolation is: ; in, Indicates the final result in ( x , y The bilinear interpolation result in the vertical direction at () Represents the ordinate of the target point; The expression for the original fused image is: ; in, Represents the original fused image. … Different feature maps are represented by N It is composed of horizontally joined two-dimensional matrices, each with dimensions of 1. M × K .

[0012] Furthermore, the image segmentation convolutional neural network includes: The coding layer is used to encode the original fused image to obtain the encoded result: ; in, Indicates the first l The output fused feature map of the layer coding layer This indicates a max pooling operation. express Activation function This indicates a 3×3 convolution kernel operation. Indicates input to the first l The fused feature map of the layer coding layer; The decoding layer is used to perform convolution and activation processing on the encoded result to obtain the decoded result: ; in, Indicates the first k The output fused feature map of the layer decoding layer. Represents the join function. This represents a 2×2 transposed convolution. Indicates that it comes from the decoding layer. k +1 layer input fused feature map, Indicates the encoder's first... k Layer fusion feature map; The output layer is used to analyze the decoding results to obtain a polarization fusion feature information image with human activity change detection results: ; in, This represents the pixel category probability matrix, i.e., the polarization fusion feature information image. express function, Represents a 1×1 convolution. This represents the output fused feature map of the last layer of the decoding layer.

[0013] Furthermore, the expression for the loss function of the image segmentation convolutional neural network is: ; in, This represents the result of the loss function. C represents the total number of pixels, and C represents the number of categories. This represents the true category label of pixel i. This represents the probability that pixel i belongs to category c, as predicted by the image segmentation convolutional neural network.

[0014] The beneficial effects of this invention are as follows: A method for detecting human activity changes based on multi-information fusion images, using original images of high-intensity seismic activity areas and polarization information images, performs deep learning-based change detection on different fused images. Using interpreted building change patches in high-intensity seismic activity areas as sample references, the method compares the detection effects of different images on building changes in high-intensity seismic activity areas to obtain the polarization fusion feature information image with the best performance. The intensity information change image is then fused with both the polarization fusion feature information image and the original image of the high-intensity seismic activity area. Change detection and comparative analysis are performed on the first and second fusion results obtained, and the image with the best detection effect on building changes in high-intensity seismic activity areas is determined, thus obtaining the optimal information fusion method. This multi-information fusion technology enhances the detection of change phenomena in images and improves the ability to detect building changes in high-intensity seismic activity areas. Building changes are a core manifestation of human activity changes; detecting building changes reflects changes in human activity. Attached Figure Description

[0015] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is an exemplary flowchart of a method for detecting changes in human activity based on multi-information fusion images, as shown in some embodiments of this specification. Figure 2 This is an exemplary schematic diagram illustrating the loss generated by the different division ratios of the training and test sets of high-intensity regional change patches as the model iteration number changes, according to some embodiments of this specification. Detailed Implementation

[0016] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0017] Example Figure 1 This is an exemplary flowchart illustrating a method for detecting changes in human activity based on multi-information fusion images, according to some embodiments of this specification. Figure 1 As shown, the process includes the following steps. In some embodiments, the process may be executed by a processor.

[0018] S1: Preprocess the original images of high-intensity areas containing human activities to obtain synthetic aperture radar images of the study area.

[0019] Human activities refer to activities that can objectively alter the Earth's surface physical structure and radar scattering characteristics, such as the construction or demolition of buildings. By performing physical-level polarization decomposition and feature extraction on raw image data of high-intensity areas containing the aforementioned changes, this addresses the technical problems of low detection accuracy and poor timeliness of changes caused by cloud and fog obstruction in existing remote sensing image processing technologies.

[0020] The original imagery of high-intensity seismic activity areas includes images of high-intensity seismic activity areas acquired by satellite remote sensing. For example, the original imagery of high-intensity seismic activity areas including human activity can include original synthetic aperture radar images from Gaofen-3, multispectral images from Gaofen-2, and panchromatic images; a comparison of the original imagery data of high-intensity seismic activity areas including human activity is shown in Table 1.

[0021] Table 1. Comparison of original images of high-intensity areas containing human activity

[0022] Synthetic Aperture Radar (SAR) imagery of the study area is used to analyze changes in high-intensity seismic activity within the study region.

[0023] In some embodiments, the processor can perform multi-view processing, filtering, geocoding, and radiometric calibration on the original Gaofen-3 synthetic aperture radar imagery, and obtain synthetic aperture radar imagery of the study area by cropping; wherein, the original Gaofen-3 synthetic aperture radar imagery belongs to the original imagery of high-intensity areas containing human activities.

[0024] In some embodiments, the processor can perform radiometric calibration, atmospheric correction, and orthorectification on the Gaofen-2 multispectral image to obtain an optical image of the study area; perform radiometric calibration and orthorectification on the panchromatic image, fuse the panchromatic and multispectral data, and obtain optical patch samples of high-intensity regional changes in the study area through visual interpretation; the original Gaofen-3 synthetic aperture radar image, the Gaofen-2 multispectral image, and the panchromatic image belong to the original images of high-intensity regions containing human activities, while the synthetic aperture radar image of the study area, the optical image of the study area, and the optical patch samples of high-intensity regional changes in the study area belong to the preprocessed remote sensing image of the study area.

[0025] The preprocessed remote sensing images of the study area were used as comparative experimental data to verify the detection effect.

[0026] S2: Perform polarization synthesis and polarization decomposition on the synthetic aperture radar image of the study area to obtain a polarization information image.

[0027] Polarimetric information images are remote sensing images of a study area that enhances the understanding of changes in the region. For example, polarimetric information images can include polarization entropy, mean scattering angle, and anisotropy.

[0028] Polarization entropy is used to distinguish the randomness of scattering.

[0029] The average scattering angle is used to characterize the dominant mechanism.

[0030] Anisotropy is used to assist in target classification.

[0031] In some embodiments, the processor can use PolSARpro software to perform polarization synthesis and polarization decomposition operations on the ground physical information acquired by the Gaofen-3 satellite under different polarization combinations to obtain a polarization information image with three parameters (polarization entropy, average scattering angle, and anisotropy).

[0032] SAR amplitude provides a radiometric reference. All features are normalized and spatially registered before being stitched together at the channel level to ensure feature complementarity in deformed regions.

[0033] S3: Extract intensity information from the synthetic aperture radar image of the study area to obtain an image showing the change in intensity information.

[0034] An intensity information change image is an image that reflects changes in the intensity information of a SAR image.

[0035] In some embodiments, the processor can divide the synthetic aperture radar image of the study area into K classes based on image intensity information to obtain multiple cluster centers; calculate the membership degree between the synthetic aperture radar image of the study area and the cluster centers; extract intensity information from the synthetic aperture radar image of the study area to obtain an intensity information change image.

[0036] In some embodiments, the processor can divide synthetic aperture radar (SAR) images into K classes based on image intensity information using a fuzzy C-means clustering method, obtaining multiple representative cluster centers that reflect the typical intensity characteristics of ground features; calculate the membership degree between the SAR images of the study area and the cluster centers, establish the association between pixels and cluster centers, extract intensity information from the SAR images of the study area, and obtain an intensity information change image by comparing and analyzing the membership degree results of the two temporal images.

[0037] In some embodiments, the processor can utilize a fuzzy C-means clustering algorithm based on differential evolution (FCM differential algorithm) to divide the SAR into K classes, obtaining multiple cluster centers. K is an integer greater than 1.

[0038] Membership degree is the degree to which a SAR image belongs to a cluster center.

[0039] In some embodiments, the expression for membership degree can be: ; ; in, Represents sample points With cluster center membership degree Representing data points i To other cluster centers k distance, Represents sample points With cluster center distance, i Indicates the index of the data point. j This represents the index of the currently calculated cluster center, and c represents the indices of all cluster centers traversed during the summation process. k This represents the total number of clusters. m Represents the fuzzy index. Indicates the cluster center. Represents sample points, Indicates membership degree of m The power is used to weight the contribution of data points to cluster centers, where n represents the total number of sample points.

[0040] Radar imagery is generated by measuring the reflection of radar waves from a target surface. Its intensity information refers to data reflecting the intensity of target reflection or scattering, typically represented as a grayscale image. The grayscale level reflects the intensity of the reflection or the energy of the echo from the target surface; different ground features have different reflection intensities. The FCM differential algorithm can suppress SAR interferometric phase noise and preserve areas sensitive to surface deformation. Therefore, based on the different radar wave reflection characteristics of ground features before and after changes, changes can be detected by analyzing the differences in image intensity information.

[0041] S4: Based on the original images of the high-intensity region and the polarization information image, a polarization fusion feature information image is obtained through image fusion and comparison detection.

[0042] Polarization fusion feature information image is the original image of a high-intensity region after fusing polarization information, which has the best effect on detecting changes in high-intensity regions.

[0043] In some embodiments, the processor may implement S4 based on the following steps.

[0044] S410: Based on the original image of the high-intensity region and the polarization information image, an original fused image is obtained through image fusion.

[0045] The original fused image is the fused image after resampling and interpolation.

[0046] In some embodiments, the processor can, based on the original image of the high-intensity region and the polarization information image, resample the image by using the weighted average of neighboring pixels to obtain resampled pixel values; perform interpolation processing on the resampled pixel values ​​to obtain horizontal and vertical interpolation; based on the resampled pixel values, the horizontal interpolation, and the vertical interpolation, construct a coordinate linear transformation relationship to obtain geographic coordinates; and stack the geographic coordinates by band dimensions to fuse multiple single-band images into a multi-band dataset to obtain the original fused image.

[0047] In some embodiments, the processor can refine the image data using a spatial domain resampling method based on the original image of the high-intensity region and the polarization information image. Resampling is performed by calculating the weighted average of the nearest neighbor pixels around the target pixel, with the weights adaptively adjusted according to the spatial distance or similarity between pixels, resulting in smoother and more representative resampled pixel values. The resampled data is then interpolated using a bilinear interpolation algorithm, optimized in both the horizontal and vertical directions to improve the spatial continuity and geometric accuracy of the image, yielding horizontal and vertical interpolations. Based on these horizontal and vertical interpolations, a coordinate linear transformation model is constructed to establish a mapping relationship between image coordinates and geographic coordinates, obtaining geographic coordinates. The geographic coordinates are then stacked by band dimensions, and multiple images with different polarization information are fused with synthetic aperture radar (SAR) images to form a multi-band dataset, resulting in the original fused image.

[0048] The resampled pixel value is the pixel value resulting from the combined influence of the nearest neighbor pixels.

[0049] In some embodiments, the processor can use a bilinear interpolation resampling method to calculate the weighted average of the four nearest neighbor pixels of the original image and polarization information image of the high-intensity region, and obtain the resampled pixel value.

[0050] Horizontal interpolation is the interpolation of resampled pixel values ​​in the horizontal direction.

[0051] Vertical interpolation is the interpolation of resampled pixel values ​​in the vertical direction.

[0052] In some embodiments, the expression for horizontal interpolation can be: ; ; in, Indicated on the y-axis The horizontal axis on the horizontal line x The interpolation result at that point, Indicated on the y-axis The horizontal axis on the horizontal line x The interpolation result at that point, Represents the x-coordinate of the target point. , Represents the x-coordinate of a known data point. , Represents the y-coordinate of a known data point. The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value.

[0053] In some embodiments, the expression for vertical interpolation can be: ; in, Indicates the final result in ( x , y The bilinear interpolation result in the vertical direction at () Represents the ordinate of the target point.

[0054] Geographic coordinates are the pixel coordinates of an image after affine transformation.

[0055] In some embodiments, the processor can utilize affine transformation to construct a linear transformation relationship of the coordinates of the resampled interpolated pixel values ​​to obtain geographic coordinates.

[0056] In some embodiments, the expression for geographic coordinates can be: ; in, and Represents geographic coordinates, Indicates the scaling factor. Indicates the rotation and shear coefficients. Indicates the rotation and shear coefficients. Indicates the scaling factor. and This indicates the resampled pixel values ​​after interpolation. Indicates the amount of translation. This indicates the amount of translation.

[0057] In some embodiments, the expression for the original fused image can be: ; in, Represents the original fused image. … Different feature maps are represented by N It is composed of horizontally joined two-dimensional matrices, each with dimensions of 1. M × K .

[0058] S420: The original fused image is analyzed using an image segmentation convolutional neural network to obtain a polarization fusion feature information image; wherein, the image segmentation convolutional neural network is obtained through training.

[0059] In some embodiments, the processor can utilize an image segmentation convolutional neural network to analyze the original fused image, perform deep learning change detection on different fused images, use the interpreted high-intensity area change patches containing human activities as sample references, and obtain the polarization fusion feature information image with the best performance by comparing the detection effects of different images on high-intensity area changes; wherein, the image segmentation convolutional neural network is obtained through training.

[0060] Image segmentation convolutional neural networks are convolutional neural networks used to detect high-intensity regional changes in original fused images.

[0061] In some embodiments, the processor can establish four fusion models through controlled variable experiments: polarization entropy-based fusion (F_H: SAR+H), scattering angle-based fusion (F_α: SAR+α), anisotropic-guided fusion (F_A: SAR+A), and polarimetric composite image fusion (F_RGB: SAR+RGB). The performance of each model is shown in Table 2. Among them, the anisotropic feature performs best in the overall detection index. Based on the optimization results, a two-level fusion model is constructed. The anisotropic-guided fusion (F_A: SAR+A) and the FCM differential intensity information image are stacked and fused (F_AF), and comparative experiments are designed for SAR image, SAR image and intensity information image fusion (F_F: SAR+FCM). Deep learning models are trained on the information fusion images respectively.

[0062] Table 2 Performance of the Fusion Model

[0063] In some embodiments, an image segmentation convolutional neural network may include an encoding layer, a decoding layer, and an output layer.

[0064] The coding layer is used to encode the original fused image to obtain the encoded result.

[0065] The encoding result is a normalized and encoded original fused image with lower resolution and more channels.

[0066] In some embodiments, the processor can utilize the coding layer to receive the original fused image and the labeled mask data, while simultaneously normalizing the input data and outputting it; the encoder part receives the output feature map of the previous layer, extracts multi-scale ground feature features step by step through the convolutional layer, and gradually reduces the resolution through max pooling. Each encoder outputs a feature map with half the resolution and double the number of channels as the encoding result.

[0067] In some embodiments, the expression for the encoded result can be: ; in, Indicates the first l The output fused feature map of the layer coding layer This indicates a max pooling operation. express Activation function This indicates a 3×3 convolution kernel operation. Indicates input to the first l The fused feature map of the layer coding layer.

[0068] The decoding layer is used to perform convolution and activation processing on the encoded result to obtain the decoded result.

[0069] The decoding result is a feature map that is restored step by step at different resolutions.

[0070] In some embodiments, the processor can utilize the decoding layer to receive the encoding result, restore the image resolution and fuse details through transposed convolution and skip connections, and finally output a feature map with progressively restored resolution to obtain the decoding result.

[0071] In some embodiments, the expression for the decoding result can be: ; in, Indicates the first k The output fused feature map of the layer decoding layer. Represents the join function. This represents a 2×2 transposed convolution. Indicates that it comes from the decoding layer. k +1 layer input fused feature map, Indicates the encoder's first... k Layer fusion feature map.

[0072] The output layer is used to analyze the decoding results to obtain a polarization fusion feature information image with human activity change detection results.

[0073] In some embodiments, the processor can utilize the output layer to receive the feature map of the last layer of the decoder and perform class probability prediction, outputting the final segmentation result as a polarization fusion feature information image.

[0074] In some embodiments, the expression for the polarization fusion feature information image can be: ; in, This represents the pixel category probability matrix, i.e., the polarization fusion feature information image. express function, Represents a 1×1 convolution. This represents the output fused feature map of the last layer of the decoding layer.

[0075] In some embodiments, the processor may divide the original fused image into a training set and a test set at a 9:1 ratio; wherein the division of the original fused image affects the number of iterations as follows: Figure 2 As shown. Figure 2 The horizontal axis represents the number of model iterations, and the vertical axis represents the loss value.

[0076] In some embodiments, an image segmentation convolutional neural network can be trained using multiple labeled training sets. For example, multiple labeled training sets can be input into an initial image segmentation convolutional neural network. A loss function is constructed using the labels and the results of the initial image segmentation convolutional neural network. Based on the loss function, the parameters of the image segmentation convolutional neural network are iteratively updated using gradient descent or other methods. The model training is complete when preset conditions are met, resulting in a trained image segmentation convolutional neural network. These preset conditions may include loss function convergence, the number of iterations reaching a threshold, etc.

[0077] In some embodiments, the labels can be images corresponding to real polarization fusion feature information. Labels can be manually annotated.

[0078] In some embodiments, the expression for the loss function can be: ; in, This represents the result of the loss function. C represents the total number of pixels, and C represents the number of categories. This represents the true category label of pixel i. This represents the probability that pixel i belongs to category c, as predicted by the image segmentation convolutional neural network.

[0079] In some embodiments, the processor can use a test set to test the performance of the trained image segmentation convolutional neural network. Five evaluation metrics—accuracy, precision, recall, F1 score, and loss—are selected for overall model performance evaluation. Specific performance test results are shown in Table 3. The comparison of change detection results shows that the model trained on SAR images has an accuracy of 84.5%, indicating good overall performance and the ability to basically identify changed regions. This demonstrates that using SAR imagery for deep learning change detection is feasible. However, in overall detection, there are still false positives and false negatives, resulting in insufficient accuracy. The model trained after fusing intensity information (F_F detection) has the worst accuracy and loss rate, at 82.7% and 37.2%, respectively. Information fusion may lead to feature redundancy or information loss, resulting in poor detection performance, especially for river detection. During intensity information extraction, it is more sensitive to noise and cannot accurately identify changed areas, easily misidentifying non-changed areas as changed areas. Finally, the model trained by simultaneously fusing polarization and intensity information (F_AF detection) achieved the best overall performance, with an accuracy of 86.5% and a loss rate of 26.3%. The fused features can better distinguish between real change areas and noise, thereby improving detection accuracy. As can be seen in the detection results, this information fusion model can accurately identify the location of change areas, producing regular and complete outlines of the change patches, and exhibits more stable detection capabilities in complex scenes (with diverse change features).

[0080] Table 3 Performance test results of the trained image segmentation convolutional neural network

[0081] S5: The intensity information change image is fused with the polarization fusion feature information image and the original image of the high-intensity area, respectively. The first fusion result and the second fusion result are detected and compared to obtain the human activity change detection result, thus completing the human activity change detection based on multi-information fusion image.

[0082] The first fusion result is the fusion of the intensity information change image and the polarization fusion feature information image.

[0083] The second fusion result is the fusion of the intensity information change image with the original image of the high-intensity area.

[0084] The results of human activity change detection are the results of changes in high-intensity areas in synthetic aperture radar imagery.

[0085] In some embodiments, the processor can fuse the intensity information change image with the polarization fusion feature information image and the original image of the high-intensity region, respectively, to obtain a first fusion result and a second fusion result. The processor then performs detection and comparative analysis on the first and second fusion results to obtain human activity change detection results, thus completing human activity change detection based on multi-information fusion images. Specifically, the processor can compare the first and second fusion results, select the best polarization fusion feature information image, and then fuse the intensity information image into the SAR image and the best-performing polarization fusion feature information image from the previous step, respectively. Finally, a comparative analysis of the change detection effects is performed to obtain the human activity change detection results.

[0086] In some embodiments of this specification, a method for detecting human activity changes based on multi-information fusion images is provided. Based on original images of high-intensity seismic activity areas and polarization information images, deep learning-based change detection is performed on different fused images. Interpreted high-intensity seismic activity change patches are used as sample references. By comparing the detection performance of different images on high-intensity seismic activity changes, the polarization fusion feature information image with the best performance is obtained. The intensity information change image is then fused with both the polarization fusion feature information image and the original high-intensity seismic activity area image. Change detection and comparative analysis are performed on the first and second fusion results, respectively, to determine the image with the best high-intensity seismic activity change detection performance, thus obtaining the optimal information fusion method. This multi-information fusion technique enhances the detection of changes in images and improves the ability to detect changes in high-intensity seismic activity areas.

Claims

1. A method for detecting changes in human activity based on multi-information fusion images, characterized in that, include: S1: Preprocess the original images of high-intensity areas containing human activities to obtain synthetic aperture radar images of the study area; S2: Perform polarization synthesis and polarization decomposition on the synthetic aperture radar image of the study area to obtain a polarization information image; S3: Extract intensity information from the synthetic aperture radar image of the study area to obtain an image showing the change in intensity information; S4: Based on the original images of the high-intensity region and the polarization information image, a polarization fusion feature information image is obtained through image fusion and comparison detection; S5: The intensity information change image is fused with the polarization fusion feature information image and the original image of the high-intensity area, respectively. The first fusion result and the second fusion result are detected and compared. Based on the comparison analysis result, the fused image is selected, the change area is extracted, and the human activity change detection result is obtained, thus completing the human activity change detection based on the multi-information fusion image.

2. The method for detecting changes in human activity based on multi-information fusion images according to claim 1, characterized in that, S1 includes: Multi-view processing, filtering, geocoding, and radiometric calibration were performed on the original Gaofen-3 synthetic aperture radar images. The synthetic aperture radar images of the study area were obtained by cropping. Among them, the original Gaofen-3 synthetic aperture radar images belong to the original images of high-intensity areas containing human activities.

3. The method for detecting changes in human activity based on multi-information fusion images according to claim 1, characterized in that, S3 includes: Based on image intensity information, the synthetic aperture radar images of the study area were divided into K classes, resulting in multiple cluster centers. The membership degrees of the synthetic aperture radar image of the study area and the cluster center are calculated, and the intensity information of the synthetic aperture radar image of the study area is extracted to obtain an intensity information change image.

4. The method for detecting changes in human activity based on multi-information fusion images according to claim 3, characterized in that, The expression for the membership degree is: ； ； in, Represents sample points With cluster center membership degree Representing data points i To other cluster centers k distance, Represents sample points With cluster center distance, i Indicates the index of the data point. j This represents the index of the currently calculated cluster center, and c represents the indices of all cluster centers traversed during the summation process. k This represents the total number of clusters. m Represents the fuzzy index. Indicates the cluster center. Represents sample points, Indicates membership degree of m The power is used to weight the contribution of data points to cluster centers, where n represents the total number of sample points.

5. The method for detecting changes in human activity based on multi-information fusion images according to claim 1, characterized in that, S4 includes: S410: Based on the original image of the high-intensity area and the polarization information image, an original fused image is obtained through image fusion; S420: The original fused image is analyzed using an image segmentation convolutional neural network to obtain a polarization fusion feature information image; wherein, the image segmentation convolutional neural network is obtained through training.

6. The method for detecting changes in human activity based on multi-information fusion images according to claim 5, characterized in that, S410 includes: Based on the original image of the high-intensity area and the polarization information image, the resampled pixel value is obtained by resampling using the weighted average of the nearest neighbor pixels; The resampled pixel values ​​are interpolated to obtain horizontal and vertical interpolation values; Based on the resampled pixel values, the horizontal interpolation, and the vertical interpolation, geographic coordinates are obtained by constructing a coordinate linear transformation relationship; The geographic coordinates are stacked by band dimensions, and multiple single-band images are fused into a multi-band dataset to obtain the original fused image.

7. The method for detecting changes in human activity based on multi-information fusion images according to claim 6, characterized in that, The expression for the horizontal interpolation is: ； ； in, Indicated on the y-axis The horizontal axis on the horizontal line x The interpolation result at that point, Indicated on the y-axis The horizontal axis on the horizontal line x The interpolation result at that point, Represents the x-coordinate of the target point. , Represents the x-coordinate of a known data point. , Represents the y-coordinate of a known data point. The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value, The coordinates of the neighboring cells are The corresponding value; The expression for the vertical interpolation is: ； in, Indicates the final result in ( x , y The bilinear interpolation result in the vertical direction at () Represents the ordinate of the target point; The expression for the original fused image is: ； in, Represents the original fused image. … Different feature maps are represented by N It is composed of horizontally joined two-dimensional matrices, each with dimensions of 1. M × K .

8. The method for detecting changes in human activity based on multi-information fusion images according to claim 5, characterized in that, The image segmentation convolutional neural network includes: The coding layer is used to encode the original fused image to obtain the encoded result: ； in, Indicates the first l The output fused feature map of the layer coding layer This indicates a max pooling operation. express Activation function This indicates a 3×3 convolution kernel operation. Indicates input to the first l The fused feature map of the layer coding layer; The decoding layer is used to perform convolution and activation processing on the encoded result to obtain the decoded result: ； in, Indicates the first k The output fused feature map of the layer decoding layer. Represents the join function. This represents a 2×2 transposed convolution. Indicates that it comes from the decoding layer. k +1 layer input fused feature map, Indicates the encoder's first... k Layer fusion feature map; The output layer is used to analyze the decoding results to obtain a polarization fusion feature information image with human activity change detection results: ； in, This represents the pixel category probability matrix, i.e., the polarization fusion feature information image. express function, Represents a 1×1 convolution. This represents the output fused feature map of the last layer of the decoding layer.

9. The method for detecting changes in human activity based on multi-information fusion images according to claim 5, characterized in that, The expression for the loss function of the image segmentation convolutional neural network is: ； in, This represents the result of the loss function. C represents the total number of pixels, and C represents the number of categories. This represents the true category label of pixel i. This represents the probability that pixel i belongs to category c, as predicted by the image segmentation convolutional neural network.

Images