Method and device for determining main direction of remote sensing image

Abstract

The invention discloses a remote sensing image main direction determination method and device, and relates to the technical field of remote sensing image processing, and the method comprises the steps: constructing a remote sensing direction determination network model based on a convolutional neural network model; based on the fused multi-source same-region remote sensing image, a network model is determined through a remote sensing direction, and a remote sensing image local region direction tendency probability value is obtained; based on the remote sensing image local region direction tendency probability value, analyzing the direction distribution of the remote sensing image local region under each scale through a multi-scale analysis method to construct a multi-scale pyramid structure, and obtaining a remote sensing image global region direction distribution curve; based on the remote sensing image global region direction distribution curve, consistency verification is carried out through a cross-scale consistency verification method, and the main direction of the remote sensing image is obtained. Different source image information is fused through the IHS transformation algorithm, the problem of difference between image data sources is effectively solved, and the accuracy and reliability of determining the main direction of the remote sensing image are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and particularly to a method and device for determining the main direction of a remote sensing image. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images have been widely used in fields such as environmental monitoring, land use, and urban planning. The determination of the main direction of a remote sensing image, as a key step in remote sensing image processing, is crucial for improving image quality and accuracy. In recent years, with the progress of multi-source data acquisition means, remote sensing image fusion technologies with different resolutions and different bands have been widely studied and applied. In particular, the IHS transformation algorithm can effectively fuse data from different sources and improve the spatial resolution and spectral information of remote sensing images by separately processing the brightness, chromaticity, and saturation information of the images. In addition, as a deep learning method, the convolutional neural network (CNN) has achieved remarkable results in the fields of remote sensing image classification, target detection, and main direction recognition due to its powerful feature extraction ability. Therefore, determining the main direction based on multi-source remote sensing image data has become a hot topic in current technical research.

[0003] However, the existing methods for determining the main direction of remote sensing images still have certain limitations. First, most of the existing technologies analyze remote sensing images from a single source, ignoring the complementary advantages between multi-source remote sensing images. This results in an unsatisfactory effect of image information fusion when processing high-resolution remote sensing data, thereby affecting the accuracy of the main direction. Second, although the convolutional neural network is widely used in remote sensing image processing, there is still a lack of a unified processing framework for image feature extraction and direction analysis at different scales, resulting in low accuracy of the direction tendency probability values in local regions and being unable to effectively construct an accurate direction distribution for the global region. To address these problems, the present invention combines multi-source remote sensing image data fusion with a convolutional neural network, and uses a multi-scale analysis method and cross-scale consistency verification technology to effectively improve the accuracy and reliability of determining the main direction of remote sensing images. By using the IHS transformation algorithm for multi-source data fusion, not only the problem of image information complementarity is solved, but also the image features can be refined at different scales, thereby improving the accuracy and adaptability of determining the main direction of remote sensing images. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for determining the main direction of a remote sensing image, which solves the problems of insufficient multi-source image information fusion and low accuracy of scale feature extraction.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for determining the main direction of remote sensing images, which includes collecting multi-source remote sensing images of the same region and preprocessing the multi-source remote sensing images of the same region; converting the RGB color space to the IHS color space through the IHS transformation algorithm, performing multi-source data fusion, and obtaining the fused multi-source remote sensing images of the same region; constructing a remote sensing direction determination network model based on a convolutional neural network model; obtaining the direction tendency probability values of local regions of the remote sensing images based on the fused multi-source remote sensing images of the same region through the remote sensing direction determination network model; analyzing the direction distribution of local regions of the remote sensing images at each scale through a multi-scale analysis method to construct a multi-scale pyramid structure, and obtaining the global region direction distribution curve of the remote sensing images; and performing consistency verification through a cross-scale consistency verification method based on the global region direction distribution curve of the remote sensing images to obtain the main direction of the remote sensing images.

[0008] As a preferred solution of the method for determining the main direction of remote sensing images according to the present invention, wherein: the multi-source remote sensing images of the same region include optical remote sensing regional remote sensing images, radar SAR regional remote sensing images, and LiDAR regional remote sensing images;

[0009] The preprocessing of the multi-source remote sensing images of the same region includes radiometric correction, geometric correction, and image registration.

[0010] As a preferred solution of the method for determining the main direction of remote sensing images according to the present invention, wherein: the specific steps of converting the RGB color space to the IHS color space through the IHS transformation algorithm, performing multi-source data fusion, and obtaining the fused multi-source remote sensing images of the same region are as follows,

[0011] Based on the preprocessed optical remote sensing regional remote sensing images, perform IHS transformation through the IHS transformation algorithm to convert the RGB color space to the IHS color space, and obtain the brightness component, hue component, and saturation component of the optical image after IHS transformation;

[0012] Based on the preprocessed radar SAR regional remote sensing images, convert the radar SAR regional remote sensing images into the backscattering coefficient of remote sensing images through a radiometric calibration method;

[0013] Based on the preprocessed LiDAR regional remote sensing images, convert the LiDAR regional remote sensing image point cloud data into a regular grid through the DSM method to obtain the elevation information of the remote sensing images;

[0014] Through normalization processing and linear transformation, adjust the backscattering coefficient of the remote sensing images and the elevation information of the remote sensing images to the same dynamic range as the brightness component of the optical image;

[0015] The backscattering coefficient of the processed remote sensing image and the elevation information of the remote sensing image are weighted and fused with the brightness component of the optical image to obtain a new brightness component;

[0016] The new brightness component is combined with the hue component and the saturation component through the inverse IHS transformation, and the IHS color space is reconverted back to the RGB color space to obtain a fused multi-source remote sensing image of the same region.

[0017] As a preferred embodiment of the method for determining the main direction of the remote sensing image according to the present invention, wherein: based on the convolutional neural network model, a remote sensing direction determination network model is constructed, and the specific steps are as follows:

[0018] Using the convolutional neural network model as the basic model;

[0019] The input layer receives the fused historical multi-source remote sensing images of the same region;

[0020] The convolutional layer gradually extracts the feature maps of the fused historical multi-source remote sensing images of the same region by adding multiple depthwise separable convolutional layers;

[0021] The pooling layer reduces the spatial dimension of the feature maps by adding a max pooling layer after each depthwise separable convolutional layer;

[0022] The fully connected layer performs a higher-level abstraction on the feature maps processed by the convolutional layer and the pooling layer;

[0023] The output layer outputs the final result;

[0024] Finally, a remote sensing direction determination network model is constructed.

[0025] As a preferred embodiment of the method for determining the main direction of the remote sensing image according to the present invention, wherein: based on the fused multi-source remote sensing images of the same region, the direction tendency probability value of the local region of the remote sensing image is obtained through the remote sensing direction determination network model, and the specific steps are as follows:

[0026] Through the remote sensing direction determination network model, the texture direction features and spatial structure information of the local region of the remote sensing image are extracted by using depthwise separable convolution, the direction tendency of the local region in the remote sensing image is determined by combining the attention mechanism, and a high-level abstract representation is completed through the fully connected layer, and finally the direction tendency probability value of the local region of the remote sensing image is obtained, and the expression is:

[0027]

[0028] Among them, O is the directional tendency probability value of the local area of the remote sensing image, σ is the Sigmoid function, η is the overall proportional adjustment factor, C is the conversion of the processed feature map into a high-level abstract representation, A is the attention mechanism operation, M is the max pooling operation, D(F; K) is the result obtained by performing a depthwise separable convolution operation on the fused multi-source same-region remote sensing image F through the weight parameter K of the depthwise separable convolution layer, D is the depthwise separable convolution operation, F is the fused multi-source same-region remote sensing image, K is the weight parameter of the depthwise separable convolution layer, W is the weight parameter of the fully connected layer, λ is the constant term adjustment factor in the denominator, μ is the adjustment factor of the absolute value square term coefficient, and ν is the adjustment factor of the power exponent.

[0029] As a preferred solution of the method for determining the main direction of the remote sensing image described in the present invention, wherein: based on the directional tendency probability value of the local area of the remote sensing image, a multi-scale pyramid structure is constructed by analyzing the directional distribution of the local area of the remote sensing image at each scale through a multi-scale analysis method, and the global area directional distribution curve of the remote sensing image is obtained. The specific steps are as follows:

[0030] Based on the fused multi-source same-region remote sensing image, an initial image of the pyramid structure is constructed;

[0031] The resolution of the initial image of the pyramid structure is reduced through the Gaussian blur method and the downsampling method, and the pyramid images are constructed layer by layer to obtain a multi-scale pyramid structure;

[0032] Based on the multi-scale pyramid structure, the gradient direction and gradient magnitude of the local area of the remote sensing image at each scale are analyzed through the Sobel operator;

[0033] Combined with the directional tendency probability value of the local area of the remote sensing image, the directional distribution of the local area of the remote sensing image at each scale is extracted through the sliding window method, and the directional distribution histogram of the local area of the remote sensing image at each scale is obtained;

[0034] Through the multi-scale analysis method, the directional distribution histograms of the local areas of the remote sensing image at each scale are combined to obtain the directional spectrum value of the global area directional distribution curve of the remote sensing image. The expression is:

[0035]

[0036] where P i is the directional spectrum value of the global area directional distribution curve of the remote sensing image at the i-th scale, w i is the weight coefficient at the i-th scale, f i is the weighted function at the i-th scale, h i (θ) is the histogram in the θ direction at the i-th scale, θ is the direction angle, g i is the local smoothing function at the i-th scale, h iis the direction histogram at the i-th scale, where i is the index variable of the scale and N is the number of scales;

[0037] By using the interpolation method, the directional spectrum values of the global regional direction distribution curve of the remote sensing image at each scale are connected to form the global regional direction distribution curve of the remote sensing image.

[0038] As a preferred scheme of the method for determining the main direction of the remote sensing image according to the present invention, wherein: based on the global regional direction distribution curve of the remote sensing image, consistency verification is performed by using the cross-scale consistency verification method to obtain the main direction of the remote sensing image. The specific steps are as follows.

[0039] By using the cross-scale consistency verification method, calculate the consistency metric value of the global regional direction distribution curve of the remote sensing image at each scale. The expression is:

[0040]

[0041] where ρ(P i , P j ) is the consistency metric value between the global regional direction distribution curve of the remote sensing image at the i-th scale and the global regional direction distribution curve of the remote sensing image at the j-th scale. P i is the global regional direction distribution curve of the remote sensing image at the i-th scale, and P j is the global regional direction distribution curve of the remote sensing image at the j-th scale. μ i is the mean of the direction distribution at the i-th scale, and μ j is the mean of the direction distribution at the j-th scale. j is the index variable of another scale;

[0042] According to the historical image direction judgment result, analyze by using the statistical analysis method to set the image direction threshold Ф;

[0043] Compare the consistency metric value of the global regional direction distribution curve of the remote sensing image with the image direction threshold;

[0044] When ρ(P i , P j ) > Ф, it indicates that the global regional direction distributions of the remote sensing images between different scales are consistent, and the main direction of the remote sensing image is determined;

[0045] When ρ(P i , P j ) ≤ Ф, it indicates that the global regional direction distributions of the remote sensing images between different scales lack consistency, and the main direction of the remote sensing image cannot be determined.

[0046] In a second aspect, the present invention provides a device for determining the main direction of remote sensing images, comprising a data collection module, a data fusion module, a model construction module, a model output module, a multi-scale analysis module, and a main direction determination module; the data collection module is configured to collect multi-source remote sensing images of the same region and preprocess the multi-source remote sensing images of the same region; the data fusion module is configured to convert the RGB color space into the IHS color space through the IHS transformation algorithm, perform multi-source data fusion, and obtain fused multi-source remote sensing images of the same region; the model construction module is configured to construct a remote sensing direction determination network model based on a convolutional neural network model; the model output module is configured to obtain the direction tendency probability value of the local region of the remote sensing image based on the fused multi-source remote sensing images of the same region through the remote sensing direction determination network model; the multi-scale analysis module is configured to analyze the direction distribution of the local region of the remote sensing image at each scale through a multi-scale analysis method based on the direction tendency probability value of the local region of the remote sensing image, construct a multi-scale pyramid structure, and obtain the global region direction distribution curve of the remote sensing image; the main direction determination module is configured to perform consistency verification through a cross-scale consistency verification method based on the global region direction distribution curve of the remote sensing image to obtain the main direction of the remote sensing image.

[0047] In a third aspect, the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and: when the computer program is executed by the processor, any step of the method for determining the main direction of remote sensing images as described in the first aspect of the present invention is implemented.

[0048] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the method for determining the main direction of remote sensing images as described in the first aspect of the present invention is implemented.

[0049] The beneficial effects of the present invention are as follows: By fusing different source image information through the IHS transformation algorithm, the problem of differences between image data sources is effectively solved, providing more consistent and high-quality image data for subsequent analysis; at the same time, the convolutional neural network model combined with the attention mechanism accurately extracts the direction features in the remote sensing image, optimizes the calculation of the direction tendency probability value of the local region, ensures the accurate identification of the main direction of the image, and significantly improves the accuracy and reliability of determining the main direction of the remote sensing image. Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0051] Figure 1Flowchart of the method for determining the main direction of remote sensing images in Embodiment 1.

[0052] Figure 2 Schematic diagram of the device for determining the main direction of remote sensing images in Embodiment 1. Specific implementation manners

[0053] To make the above objects, features, and advantages of the present invention more obvious and understandable, the specific implementation manners of the present invention will be described in detail below with reference to the accompanying drawings of the specification.

[0054] In the following description, many specific details are set forth to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0055] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or alternative embodiments that exclude each other with other embodiments.

[0056] Embodiment 1, referring to Figure 1 and Figure 2 , is the first embodiment of the present invention. This embodiment provides a method for determining the main direction of remote sensing images, including the following steps:

[0057] S1. The multi-source remote sensing images of the same region include optical remote sensing regional remote sensing images, radar SAR regional remote sensing images, and LiDAR regional remote sensing images.

[0058] S1.1. Preprocess the multi-source remote sensing images of the same region, including radiometric correction, geometric correction, and image registration.

[0059] It should be noted that the specific process of preprocessing the multi-source remote sensing images of the same region through radiometric correction, geometric correction, and image registration is as follows:

[0060] Radiometric correction: First, convert the digital numerical value (DN value) of the original image into radiance value or radiance ratio using the known sensor calibration parameters, and then estimate and remove the influence of atmospheric scattering and absorption according to the atmospheric conditions (such as visibility, atmospheric type, etc.) at the time of image acquisition by using techniques such as simplified empirical linear regression method or dark object subtraction method.

[0061] Geometric correction: First, select a geographic coordinate system and identify multiple ground control points (GCPs) on the image to be corrected. The position information of these points needs to be known high-precision geographic coordinates. Then, use these ground control points and adopt polynomial transformation to calculate the transformation parameters from the image coordinate system to the geographic coordinate system. Next, based on the calculated transformation parameters, resample the image to be corrected. Usually, methods such as the nearest neighbor method, bilinear interpolation, or cubic convolution are used to generate a new image file, ensuring that each pixel corresponds to the correct geographic location and achieving precise spatial alignment between images from different sources.

[0062] Image registration: First, select an image as the reference image and choose multiple homologous points with obvious features, easy to identify, and stable positions (such as road intersections, building corners, etc.) in all images to be registered. These points should be clearly located on both the reference image and the images to be registered. Then, based on these homologous points, calculate the spatial transformation relationship between the images to be registered and the reference image. Common methods include affine transformation, projective transformation, etc., to determine the rotation, scaling, and translation parameters. Next, use the above transformation parameters to resample the images to be registered. Common resampling methods include the nearest neighbor method, bilinear interpolation, or cubic convolution method to generate a new image file, making its geographic location information match precisely with the reference image.

[0063] S2. Convert the RGB color space to the IHS color space through the IHS transformation algorithm for multi-source data fusion to obtain fused multi-source remote sensing images of the same region.

[0064] S2.1. Based on the preprocessed optical remote sensing regional remote sensing image, perform IHS transformation through the IHS transformation algorithm to convert the RGB color space to the IHS color space, and obtain the brightness component, hue component, and saturation component of the optical image after IHS transformation.

[0065] It should be noted that first, apply a linear transformation to the RGB values of each pixel to calculate the brightness component, which is the weighted average of the red, green, and blue channels, and the weights may vary according to the specific sensor characteristics. Then, use the ratio relationship between the RGB values and calculate the hue component through the arctangent function, which represents the position of the color on the color wheel. Next, the saturation component is a measure based on color purity, and the calculation method is the ratio of the difference between the maximum and minimum color intensities to the brightness component. In this way, the RGB values of each pixel are converted into the corresponding brightness, hue, and saturation values, completing the conversion from the RGB to the IHS color space.

[0066] It also should be noted that during the entire conversion process, the RGB input values of each pixel are precisely mapped to the corresponding values in the IHS color space, thereby obtaining the brightness, hue, and saturation components of the optical image.

[0067] S2.2. Based on the preprocessed radar SAR regional remote sensing images, convert the radar SAR regional remote sensing images into the backscattering coefficient of the remote sensing images through the radiometric calibration method.

[0068] It should be noted that, first, obtain the sensor-specific calibration parameters according to the metadata of the SAR images, including information such as gain and attenuation. Then, apply these calibration parameters to correct the digital numerical (DN) values of the original images to compensate for the gain and bias errors inside the system. Next, further adjust the radar SAR regional remote sensing images by using the known ground object reflection characteristics to eliminate the errors caused by factors such as atmospheric influence and surface tilt. Subsequently, according to the radiometric calibration mathematical model (such as the sigma-naught model), convert the corrected DN values into the backscattering coefficient in physical sense.

[0069] It also should be noted that the known ground object reflection characteristics refer to the backscattering coefficient or reflectivity of electromagnetic waves of specific ground objects (such as soil, vegetation, water bodies, etc.) at different bands or frequencies, which are usually obtained through on-site measurement, laboratory analysis, and long-term scientific research accumulation.

[0070] S2.3. Based on the preprocessed LiDAR regional remote sensing images, convert the LiDAR regional remote sensing image point cloud data into a regular grid through the DSM method to obtain the elevation information of the remote sensing images.

[0071] It should be noted that, first, filter and classify the LiDAR point cloud data to remove noise and distinguish ground points from non-ground points. Then, use interpolation algorithms (such as inverse distance weighting method IDW, Kriging method, or natural neighbor method, etc.) to generate regularly spaced grids based on the ground points, and the value of each grid cell represents the ground elevation at that position.

[0072] S2.4. Through normalization processing and linear transformation, based on the backscattering coefficient of the remote sensing images and the elevation information of the remote sensing images, adjust to the same dynamic range as the brightness component of the optical image.

[0073] It should be noted that, first, normalize the backscattering coefficient and elevation data of the remote sensing images so that their normalized values are mapped to a standard range, usually between 0 and 1. Then, traverse all the pixels of the optical image, record the brightness value of each pixel, determine the minimum and maximum values among these brightness values to define the dynamic range of the brightness component. Then, according to this dynamic range, adjust the normalized backscattering coefficient and elevation information through linear transformation so that their normalized values can match the dynamic range of the brightness component of the optical image.

[0074] S2.5. Perform weighted fusion on the processed backscattering coefficient of the remote sensing images and the elevation information of the remote sensing images with the brightness component of the optical image to obtain a new brightness component.

[0075] It should be noted that first, the weights of the backscattering coefficient of the remote sensing image, the elevation information of the remote sensing image, and the brightness component of the optical image are determined. These weights can be set according to the importance and contribution degree of the data. Then, for each pixel position, using its corresponding backscattering coefficient, elevation information, and optical image brightness component, multiply their respective values according to the preset weight ratio. Next, add the results of these three weighted values to obtain the new brightness value of this pixel position.

[0076] S2.6. Combine the new brightness component with the hue component and saturation component through the inverse IHS transform, and re-convert the IHS color space back to the RGB color space to obtain the fused multi-source same-region remote sensing image.

[0077] It should be noted that first, using the adjusted new brightness component, the original hue component, and saturation component, calculate the new values of the red, green, and blue channels according to the inverse transform formula from IHS to RGB, decode the brightness, hue, and saturation information back to the original color space representation, and finally generate the fused multi-source same-region remote sensing image that integrates multi-source information (such as backscattering coefficient and elevation data).

[0078] S3. Based on the convolutional neural network model, construct a remote sensing direction determination network model.

[0079] S3.1. Use the convolutional neural network model as the basic model;

[0080] The input layer receives the fused historical multi-source same-region remote sensing image;

[0081] The convolutional layer gradually extracts the feature map of the fused historical multi-source same-region remote sensing image by adding multiple depthwise separable convolutional layers;

[0082] The pooling layer reduces the spatial dimension of the feature map by adding a max-pooling layer after each depthwise separable convolutional layer;

[0083] The fully connected layer performs a higher-level abstraction on the feature map processed by the convolutional layer and the pooling layer;

[0084] The output layer outputs the final result;

[0085] Finally, construct the remote sensing direction determination network model.

[0086] It should be noted that the advantage of constructing a remote sensing direction determination network model based on a convolutional neural network (CNN) model is that CNN can automatically and efficiently learn and extract complex feature representations from the fused multi-source remote sensing images. While enhancing the feature extraction ability through depthwise separable convolutional layers, it reduces the computational burden, uses the max pooling layer to reduce the data dimension and retain key features, and finally achieves high-level abstraction and classification via the fully connected layer, thereby effectively improving the accuracy and robustness of remote sensing image direction recognition and providing strong support for accurate direction determination.

[0087] S4. Based on the fused multi-source remote sensing images of the same region, obtain the direction tendency probability value of the local area of the remote sensing image through the remote sensing direction determination network model.

[0088] S4.1. Through the remote sensing direction determination network model, use depthwise separable convolution to extract the texture direction features and spatial structure information of the local area of the remote sensing image, combine the attention mechanism to determine the direction tendency of the local area in the remote sensing image, and complete the high-level abstract representation through the fully connected layer. Finally, obtain the direction tendency probability value of the local area of the remote sensing image. The expression is:

[0089]

[0090] where O is the direction tendency probability value of the local area of the remote sensing image, σ is the Sigmoid function, η is the overall proportional adjustment factor, C is the conversion of the processed feature map into a high-level abstract representation, A is the attention mechanism operation, M is the max pooling operation, D(F; K) is the result obtained by performing depthwise separable convolution operation on the fused multi-source remote sensing image F of the same region with the weight parameter K of the depthwise separable convolutional layer, D is the depthwise separable convolution operation, F is the fused multi-source remote sensing image of the same region, K is the weight parameter of the depthwise separable convolutional layer, W is the weight parameter of the fully connected layer, λ is the constant term adjustment factor in the denominator, μ is the adjustment factor of the absolute value square term coefficient, and ν is the adjustment factor of the power exponent.

[0091] It should be noted that the calculation process of the above expression is as follows:

[0092] First, use depthwise separable convolution operation and weight parameters to extract features from the fused multi-source remote sensing image of the same region. Then, perform max pooling operation on the result and enhance the saliency information through the attention mechanism. Next, convert the processed feature map into a high-level abstract representation and further process it in combination with the weight parameters of the fully connected layer. Subsequently, calculate a proportional adjustment factor based on these processing results using a specific formula, which contains a series of adjustment factors for fine-tuning. Finally, apply the Sigmoid function to map the proportional adjustment value to between 0 and 1 to obtain the direction tendency probability value of the local area of the remote sensing image.

[0093] S5. Based on the local area direction tendency probability values of remote sensing images, analyze the direction distribution of the local areas of remote sensing images at each scale through a multi-scale analysis method to construct a multi-scale pyramid structure, and obtain the global area direction distribution curve of the remote sensing images.

[0094] S5.1. Based on the fused multi-source remote sensing images of the same area, construct the initial image of the pyramid structure.

[0095] It should be noted that geometric correction and feature matching techniques are used to achieve precise registration between images, ensuring the consistency of the spatial positions of images from different sources. Then, using pixel-level, feature-level or decision-level fusion algorithms, select the appropriate fusion level according to application requirements, and integrate the advantageous information of each image, such as the detailed information of high-resolution images and the color information of multi-spectral images, so as to generate the initial image of the pyramid structure.

[0096] S5.2. Reduce the resolution of the initial image of the pyramid structure through the Gaussian blur method and the downsampling method, and construct the pyramid images layer by layer to obtain the multi-scale pyramid structure.

[0097] It should be noted that the Gaussian blur method is used to process the initial image to smooth the image. Subsequently, the resolution is reduced by half through downsampling (such as pixel thinning) to generate the first-layer pyramid image. Repeat the above steps of Gaussian blur and downsampling, and construct pyramid images with gradually decreasing resolutions layer by layer until the required number of layers or the minimum resolution requirement is reached, thus forming a pyramid structure containing multi-scale information.

[0098] S5.3. Based on the multi-scale pyramid structure, analyze the gradient direction and gradient magnitude of the local areas of remote sensing images at each scale through the Sobel operator.

[0099] It should be noted that first apply the Sobel operator to the pyramid images at each scale. This operator estimates the gradient value of each pixel point in the image by calculating the differences in the horizontal and vertical directions. Then, use the obtained gradient values to determine the gradient direction and gradient magnitude within the local area, where the gradient direction indicates the direction of the edge, and the gradient magnitude reflects the intensity of the edge. The entire process is repeated for each layer of images in the pyramid, so as to obtain the gradient direction and gradient magnitude of the local areas of remote sensing images at each scale.

[0100] S5.4. Combine the local area direction tendency probability values of remote sensing images, and extract the local area direction distribution of remote sensing images at each scale through the sliding window method to obtain the local area direction distribution histogram of remote sensing images at each scale.

[0101] It should be noted that the entire image is traversed using a sliding window. For each local area within the window, the angular distribution in different directions within the area is calculated based on its directional tendency probability value, and then these angular distributions are statistically analyzed to generate a directional distribution histogram for the corresponding window, representing the directional features of the local area at a specific scale. This process is repeatedly executed for each layer of the image in the pyramid structure, and finally, the directional distribution histograms of the local areas of the remote sensing image at each scale are obtained.

[0102] S5.5. Combine the directional distribution histograms of the local areas of the remote sensing image at each scale through a multi-scale analysis method to obtain the directional spectrum values of the global area directional distribution curve of the remote sensing image. The expression is as follows:

[0103]

[0104] where P i is the directional spectrum value of the global area directional distribution curve of the remote sensing image at the i-th scale, w i is the weight coefficient at the i-th scale, f i is the weighting function at the i-th scale, h i (θ) is the histogram in the θ direction at the i-th scale, θ is the direction angle, g i is the local smoothing function at the i-th scale, h i is the directional histogram at the i-th scale, i is the index variable of the scale, and N is the number of scales.

[0105] It should be noted that for each scale, first calculate the directional histogram at that scale and perform smoothing processing using the local smoothing function, then weight the smoothed histogram using the weighting function, multiply by the weight coefficient of that scale, then sum the results of all scales, and divide by the weighted sum of the smoothed histograms of all scales, so as to obtain the global area directional distribution curve of the remote sensing image at one scale.

[0106] S5.6. Connect the directional spectrum values of the global area directional distribution curves of the remote sensing image at each scale through an interpolation method to form the global area directional distribution curve of the remote sensing image.

[0107] It should be noted that based on the directional spectrum values of the global area directional distribution curves of the remote sensing image at each scale, use an interpolation method (such as linear interpolation, spline interpolation, etc.) to perform smooth transition on the directional spectrum values between different scales, fill the gaps between scales, ensure that the directional spectrum values are continuous and smooth at all scales, and finally connect these interpolated directional spectrum values in order of scale, so as to form a comprehensive global area directional distribution curve of the remote sensing image.

[0108] S6. Based on the global regional direction distribution curve of the remote sensing image, perform consistency verification through the cross-scale consistency verification method to obtain the main direction of the remote sensing image.

[0109] S6.1. Through the cross-scale consistency verification method, calculate the consistency metric value of the global regional direction distribution curve of the remote sensing image at each scale. The expression is:

[0110]

[0111] where ρ(P i , P j ) is the consistency metric value between the global regional direction distribution curve of the remote sensing image at the i-th scale and the global regional direction distribution curve of the remote sensing image at the j-th scale. P i is the global regional direction distribution curve of the remote sensing image at the i-th scale, and P j is the global regional direction distribution curve of the remote sensing image at the j-th scale. μ i is the mean of the direction distribution at the i-th scale, μ j is the mean of the direction distribution at the j-th scale, and j is the index variable of another scale.

[0112] S6.2. According to the historical image direction judgment results, perform analysis through statistical analysis methods to set the image direction threshold Ф.

[0113] It should be noted that first, collect the direction judgment results of a series of historical images, and calculate the mean and standard deviation of these results to understand the central tendency and dispersion degree of the direction changes. Then, based on the obtained statistical parameters and combined with the requirements of the specific application scenario (such as error tolerance), determine a suitable threshold.

[0114] S6.3. Compare the consistency metric value of the global regional direction distribution curve of the remote sensing image with the image direction threshold;

[0115] When ρ(P i , P j ) > Ф, it indicates that the global regional direction distribution of the remote sensing image between different scales is consistent, and determine the main direction of the remote sensing image;

[0116] When ρ(P i , P j ) ≤ Ф, it indicates that the global regional direction distribution of the remote sensing image between different scales lacks consistency, and the main direction of the remote sensing image cannot be determined.

[0117] This embodiment also provides a device for determining the main direction of a remote sensing image, including: a data collection module, a data fusion module, a model construction module, a model output module, a multi-scale analysis module, and a main direction determination module;

[0118] A data collection module for collecting multi-source remote sensing images of the same area and preprocessing the multi-source remote sensing images of the same area;

[0119] A data fusion module for converting the RGB color space to the IHS color space through the IHS transformation algorithm, performing multi-source data fusion, and obtaining fused multi-source remote sensing images of the same area;

[0120] A model construction module for constructing a remote sensing direction determination network model based on a convolutional neural network model;

[0121] A model output module for obtaining the direction tendency probability value of the local area of the remote sensing image based on the fused multi-source remote sensing images of the same area through the remote sensing direction determination network model;

[0122] A multi-scale analysis module for analyzing the direction distribution of the local area of the remote sensing image at each scale based on the direction tendency probability value of the local area of the remote sensing image, constructing a multi-scale pyramid structure, and obtaining the global area direction distribution curve of the remote sensing image;

[0123] A main direction determination module for performing consistency verification through a cross-scale consistency verification method based on the global area direction distribution curve of the remote sensing image to obtain the main direction of the remote sensing image.

[0124] This embodiment also provides a computer device applicable to the case of the method for determining the main direction of a remote sensing image, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for determining the main direction of a remote sensing image as proposed in the above embodiment.

[0125] This computer device can be a terminal. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of this computer device is used to provide computing and control capabilities. The memory of this computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of this computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of this computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of this computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0126] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for determining the main direction of remote sensing images as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disc.

[0127] In summary, the present invention: fuses different source image information through the IHS transformation algorithm, effectively solves the problem of differences between image data sources, and provides more consistent and high-quality image data for subsequent analysis; at the same time, the convolutional neural network model combines the attention mechanism, accurately extracts the direction features in the remote sensing image, optimizes the calculation of the local area direction tendency probability value, ensures the accurate recognition of the main direction of the image, and significantly improves the accuracy and reliability of determining the main direction of the remote sensing image.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for determining the main direction of a remote sensing image, characterized in that: include, Collect multi-source remote sensing images of the same region and pre-process them; The RGB color space is converted into the IHS color space through the IHS transformation algorithm, and multi-source data fusion is performed to obtain fused multi-source remote sensing images of the same area; Based on the convolutional neural network model, a remote sensing direction determination network model is constructed; Based on the fusion of multi-source remote sensing images of the same region, the network model is determined by remote sensing direction to obtain the probability value of the direction tendency of the local area of ​​the remote sensing image; Based on the probability value of the directional tendency of the local area of ​​the remote sensing image, the directional distribution of the local area of ​​the remote sensing image at each scale is analyzed by a multi-scale analysis method to construct a multi-scale pyramid structure and obtain the directional distribution curve of the global area of ​​the remote sensing image; Based on the global regional direction distribution curve of the remote sensing image, consistency verification is performed through the cross-scale consistency verification method to obtain the main direction of the remote sensing image.

2. The method for determining the main direction of a remote sensing image according to claim 1, wherein: The multi-source same-region remote sensing images include optical remote sensing regional remote sensing images, radar SAR regional remote sensing images and LiDAR regional remote sensing images; The preprocessing of multi-source remote sensing images of the same region includes radiation correction, geometric correction and image registration.

3. The method for determining the main direction of a remote sensing image according to claim 2, wherein: The RGB color space is converted into the IHS color space through the IHS transformation algorithm, multi-source data fusion is performed, and the fused multi-source remote sensing image of the same area is obtained. The specific steps are as follows: Based on the pre-processed optical remote sensing area remote sensing image, the RGB color space is converted into the IHS color space through the IHS transformation algorithm to obtain the brightness component, hue component and saturation component of the optical image after the IHS transformation; Based on the preprocessed radar SAR regional remote sensing image, the radar SAR regional remote sensing image is converted into the remote sensing image backscattering coefficient through the radiation calibration method; Based on the preprocessed LiDAR regional remote sensing image, the LiDAR regional remote sensing image point cloud data is converted into a regular grid through the DSM method to obtain the remote sensing image elevation information; Through normalization and linear transformation, the dynamic range is adjusted to the same as the brightness component of the optical image based on the backscatter coefficient of the remote sensing image and the elevation information of the remote sensing image; The processed remote sensing image backscattering coefficient and remote sensing image elevation information are weightedly fused with the brightness component of the optical image to obtain a new brightness component; The new brightness component is combined with the hue component and the saturation component through the inverse IHS transformation, and the IHS color space is converted back to the RGB color space to obtain the fused multi-source remote sensing image of the same area.

4. The method for determining the main direction of a remote sensing image according to claim 3, wherein: The remote sensing direction determination network model is constructed based on the convolutional neural network model. The specific steps are as follows: The convolutional neural network model is used as the basic model; The input layer receives the fused historical multi-source remote sensing images of the same region; The convolution layer gradually extracts the fused feature maps of historical multi-source remote sensing images of the same region by adding multiple depth-separable convolution layers; The pooling layer reduces the spatial dimension of the feature map by adding a max pooling layer after each depthwise separable convolutional layer; The fully connected layer abstracts the feature maps processed by the convolutional layer and the pooling layer at a higher level; The output layer outputs the final result; Finally, a remote sensing direction determination network model was constructed.

5. The method for determining the main direction of a remote sensing image according to claim 4, characterized in that: Based on the fusion of multi-source remote sensing images of the same region, the network model is determined by remote sensing direction to obtain the probability value of the direction tendency of the local area of ​​the remote sensing image. The specific steps are as follows: Through the remote sensing direction determination network model, the texture direction features and spatial structure information of the local area of ​​the remote sensing image are extracted by using the deep separable convolution. The direction tendency of the local area in the remote sensing image is determined by combining the attention mechanism, and the high-level abstract representation is completed through the fully connected layer. Finally, the probability value of the direction tendency of the local area of ​​the remote sensing image is obtained, which is expressed as: Among them, O is the probability value of the directional tendency of the local area of ​​the remote sensing image, σ is the Sigmoid function, η is the overall scale adjustment factor, C is the conversion of the processed feature map into a high-level abstract representation, A is the attention mechanism operation, M is the maximum pooling operation, D(F; K) is the result of performing a deep separable convolution operation on the fused multi-source remote sensing image F of the same region through the weight parameter K of the deep separable convolution layer, D is the deep separable convolution operation, F is the fused multi-source remote sensing image of the same region, K is the weight parameter of the deep separable convolution layer, W is the weight parameter of the fully connected layer, λ is the constant term adjustment factor in the denominator, μ is the adjustment factor of the absolute value square term coefficient, and ν is the adjustment factor of the power exponent.

6. The method for determining the main direction of a remote sensing image according to claim 5, characterized in that: Based on the probability value of the directional tendency of the local area of ​​the remote sensing image, a multi-scale analysis method is used to analyze the directional distribution of the local area of ​​the remote sensing image at each scale to construct a multi-scale pyramid structure, and obtain the directional distribution curve of the global area of ​​the remote sensing image. The specific steps are as follows: Based on the fusion of multi-source remote sensing images of the same area, the initial image of the pyramid structure is constructed; The resolution of the initial image of the pyramid structure is reduced by using the Gaussian blur method and the downsampling method, and the pyramid image is constructed layer by layer to obtain a multi-scale pyramid structure. Based on the multi-scale pyramid structure, the Sobel operator is used to analyze the gradient direction and gradient amplitude of the local area of ​​the remote sensing image at each scale; Combined with the directional tendency probability value of the local area of ​​the remote sensing image, the directional distribution of the local area of ​​the remote sensing image of each scale is extracted through the sliding window method to obtain the directional distribution histogram of the local area of ​​the remote sensing image of each scale; The directional distribution histograms of the local area of ​​the remote sensing image at each scale are combined through a multi-scale analysis method to obtain the directional spectrum value of the directional distribution curve of the global area of ​​the remote sensing image. The expression is: Among them, P i is the directional spectrum value of the global regional directional distribution curve of the remote sensing image at the i-th scale, w i is the weight coefficient of the i-th scale, f i is the weighting function of the i-th scale, h i (θ) is the θ direction histogram at the i-th scale, θ is the direction angle, g i is the local smoothing function of the i-th scale, h i is the direction histogram at the i-th scale, i is the index variable of the scale, and N is the number of scales; The directional spectrum values ​​of the global regional directional distribution curve of the remote sensing image at each scale are connected by the interpolation method to form the global regional directional distribution curve of the remote sensing image.

7. The method for determining the main direction of a remote sensing image according to claim 6, characterized in that: The global regional direction distribution curve of the remote sensing image is used to perform consistency verification through a cross-scale consistency verification method to obtain the main direction of the remote sensing image. The specific steps are as follows: Through the cross-scale consistency verification method, the consistency measurement value of the global regional direction distribution curve of the remote sensing image at each scale is calculated, and the expression is: Among them, ρ(P i ,P j ) is the consistency measure of the global regional direction distribution curve of the remote sensing image at the i-th scale and the global regional direction distribution curve of the remote sensing image at the j-th scale, P i is the global regional direction distribution curve of the remote sensing image at the i-th scale, P j is the global regional direction distribution curve of the remote sensing image at the jth scale, μ i is the mean of the distribution in the i-th scale direction, μ j is the mean of the distribution in the jth scale direction, and j is the index variable of another scale; According to the results of historical image direction judgment, the image direction threshold Ф is set by statistical analysis method; Compare the consistency measure of the global regional direction distribution curve of the remote sensing image with the image direction threshold; When ρ(P i ,P j )>Ф, it indicates that the global regional direction distribution of remote sensing images of different scales is consistent, and the main direction of the remote sensing image is determined; When ρ(P i ,P j )≤Ф, it indicates that the global regional direction distribution of remote sensing images of different scales lacks consistency and the main direction of the remote sensing image cannot be determined.

8. A remote sensing image main direction determination device, based on the remote sensing image main direction determination method according to any one of claims 1 to 7, characterized in that: It includes a data collection module, a data fusion module, a model building module, a model output module, a multi-scale analysis module and a main direction determination module; The data collection module is used to collect multi-source remote sensing images of the same region and pre-process the multi-source remote sensing images of the same region; The data fusion module is used to convert the RGB color space into the IHS color space through the IHS transformation algorithm, perform multi-source data fusion, and obtain fused multi-source remote sensing images of the same area; A model building module is used to build a remote sensing direction determination network model based on a convolutional neural network model; The model output module is used to determine the network model based on the fusion of multi-source remote sensing images of the same region and obtain the probability value of the direction tendency of the local area of ​​the remote sensing image; The multi-scale analysis module is used to analyze the directional distribution of the local area of ​​the remote sensing image at each scale through a multi-scale analysis method based on the directional tendency probability value of the local area of ​​the remote sensing image to construct a multi-scale pyramid structure and obtain the directional distribution curve of the global area of ​​the remote sensing image; The main direction determination module is used to obtain the main direction of the remote sensing image by performing consistency verification based on the global regional direction distribution curve of the remote sensing image through a cross-scale consistency verification method.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for determining the main direction of a remote sensing image according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for determining the main direction of a remote sensing image according to any one of claims 1 to 7 are implemented.

Images