Remote sensing mechanism depth coupling remote sensing image building detection method

Abstract

This application provides a remote sensing image building detection method with deep coupling of remote sensing mechanisms, relating to the field of remote sensing image processing technology. The detection method includes: acquiring multiple remote sensing images containing the target building, along with the location information and imaging parameters of each remote sensing image; performing geometric correction and coordinate unification on the multiple remote sensing images based on the location information to obtain a standardized remote sensing image set; slicing the standardized remote sensing image set according to geographical range and resolution to obtain an initial image slice set; constructing a sample index library based on the imaging parameters, setting imaging screening rules based on the sample index library, and selecting the target image slice set from the initial image slice set according to the imaging screening rules; extracting deep features of each target image slice from the target image slice set through an image backbone network, fusing the imaging parameters and deep features to obtain fused features, and determining the category and location information of the target building based on the fused features.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, and in particular to a method for detecting buildings in remote sensing images that is deeply coupled with remote sensing mechanisms. Background Technology

[0002] With the rapid development of remote sensing imaging technology and high-resolution satellite imagery, remote sensing images have been widely used in urban planning, land use monitoring, disaster assessment, and other fields. Among these applications, automatic building detection has become one of the important tasks in intelligent remote sensing interpretation. Existing building detection methods mostly rely on deep learning models to automatically learn and recognize pixel or texture features in remote sensing images, achieving high accuracy and processing efficiency.

[0003] However, traditional methods still have certain limitations in sample construction and model generalization. On the one hand, due to the diverse sources of remote sensing images and their significant influence from factors such as imaging time, shooting angle, and geographical location, there are large differences in appearance between different images, leading to inconsistent distribution of training samples and thus affecting model performance. On the other hand, existing methods often ignore the imaging parameter information attached to remote sensing images, such as pitch angle, roll angle, and imaging latitude. These parameters, as important reflections of imaging geometry and lighting conditions, actually have a significant impact on the appearance and recognizability of buildings.

[0004] In addition, traditional sample construction methods often use image slices as the basic unit, dividing training samples only according to image content. This lacks unified management and control of imaging conditions, resulting in strong conditional biases between samples, making it difficult to guarantee detection consistency and robustness under complex spatiotemporal conditions. Summary of the Invention

[0005] In view of the above problems, this application provides a method for building detection in remote sensing images that is deeply coupled with remote sensing mechanisms.

[0006] According to a first aspect of this application, a method for detecting buildings in remote sensing images with deep coupling of remote sensing mechanisms is provided, comprising: acquiring multiple remote sensing images containing a target building, as well as the positioning information and imaging parameters of each remote sensing image; performing geometric correction and coordinate unification on the multiple remote sensing images based on the positioning information to obtain a standardized remote sensing image set; performing slicing processing on the standardized remote sensing image set according to the geographical range and resolution of the standardized remote sensing image set to obtain an initial image slice set; constructing a sample index library based on the imaging parameters of each initial image slice in the initial image slice set, setting imaging screening rules based on the sample index library, and selecting a target image slice set from the initial image slice set according to the imaging screening rules; extracting deep features of each target image slice from the target image slice set through a preset image backbone network, fusing the imaging parameters and deep features to obtain fused features, and determining the category and location information of the target building based on the fused features.

[0007] According to an embodiment of this application, obtaining the positioning information and imaging parameters of each remote sensing image includes: determining the positioning information and imaging parameters based on the metadata information attached to each remote sensing image; wherein, the imaging parameters include geographical location, imaging angle and image attributes.

[0008] According to embodiments of this application, geometric correction and coordinate unification are performed on multiple remote sensing images based on positioning information to obtain a standardized remote sensing image set. This includes: georegistering multiple remote sensing images based on positioning information using a control point matching algorithm to construct a transformation relationship between each remote sensing image and a reference map projection; performing geometric correction on each remote sensing image based on the transformation relationship using a polynomial geometric correction model or an affine / projective transformation model; and projecting the geometrically corrected remote sensing images together onto a preset map projection coordinate system to obtain a standardized remote sensing image set with the same spatial reference.

[0009] According to an embodiment of this application, a set of normalized remote sensing images is sliced ​​based on the geographic extent and resolution of the set to obtain an initial image slice set. This includes: obtaining the coordinate information, size information, and pixel information of each normalized remote sensing image from the geographic extent and resolution of the set; determining the geographic extent of each normalized remote sensing image based on the coordinate information, size information, and pixel information; and performing sliding window slicing on each normalized remote sensing image based on the geographic extent and pixel information to obtain the initial image slice set.

[0010] According to an embodiment of this application, a sample index library is constructed based on the imaging parameters of each initial image slice in the initial image slice set, including: generating a number for each initial image slice based on its location in a normalized remote sensing image; associating the number, geographical location, imaging angle, and image attributes of each initial image slice to form an index entry for the initial image slice; and forming a sample index library based on multiple index entries of the initial image slice set.

[0011] According to an embodiment of this application, selecting a target image slice set from an initial image slice set according to imaging screening rules includes: reading the geographical location, imaging angle, and image attributes of each initial image slice from a sample index library; determining whether the geographical location, imaging angle, and image attributes of each initial image slice all conform to the imaging screening rules; and taking each initial image slice that conforms to the imaging screening rules as a target image slice to form a target image slice set.

[0012] According to an embodiment of this application, the method further includes: after extracting the deep features of each target image slice from the target image slice set through a preset image backbone network, enhancing the deep features through a preset texture edge enhancement module to obtain enhanced deep features, and replacing the extracted deep features with the enhanced deep features.

[0013] According to an embodiment of this application, imaging parameters and deep features are fused to obtain fused features, and the category and location information of the target building are determined based on the fused features. The method includes: mapping imaging parameters into auxiliary feature vectors through a preset multilayer perceptron; fusing deep features and auxiliary feature vectors to obtain fused features; and performing convolution and fully connected processing on the fused features in sequence to obtain the category and location information of the target building.

[0014] According to a second aspect of this application, a remote sensing image building detection device with deep coupling of remote sensing mechanisms is provided, comprising: a parameter acquisition module for acquiring multiple remote sensing images containing a target building, as well as the positioning information and imaging parameters of each remote sensing image; an image correction module for performing geometric correction and coordinate unification on the multiple remote sensing images based on the positioning information to obtain a standardized remote sensing image set; an image slicing module for slicing the standardized remote sensing image set according to the geographical range and resolution of the standardized remote sensing image set to obtain an initial image slice set; a slice selection module for constructing a sample index library based on the imaging parameters of each initial image slice in the initial image slice set, setting imaging selection rules based on the sample index library, and selecting a target image slice set from the initial image slice set according to the imaging selection rules; and an information determination module for extracting deep features of each target image slice from the target image slice set through a preset image backbone network, fusing the imaging parameters and deep features to obtain fused features, and determining the category and location information of the target building based on the fused features.

[0015] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0016] The remote sensing image building detection method deeply coupled with remote sensing mechanism provided in this application has at least the following technical effects:

[0017] 1. This application incorporates imaging parameters from remote sensing images, such as pitch angle, roll angle, imaging time, and shooting latitude, to achieve joint modeling of image content and imaging geometry. This deep coupling mechanism enables the model to automatically compensate for geometric deformations caused by the shooting angle and accurately distinguish between the real base point and the projected area of ​​a building in scenes with complex lighting, significantly reducing recognition errors between cross-source data. This significantly improves the accuracy and robustness of building detection. This method not only effectively enhances the diversity and representativeness of samples, solving the recognition bias problem caused by the single imaging conditions and uneven spatial distribution of samples in traditional methods, but also improves the model's adaptability and performance under remote sensing images.

[0018] 2. This application achieves precise control and scientific selection of training sample distribution by constructing a joint indexing system of imaging parameters and image content. In this way, the model can learn the invariant characteristics of buildings under different physical parameters, which greatly enhances the robustness of the detection algorithm in different geographical regions and seasons, giving the system a stronger scene transfer capability.

[0019] 3. This application constructs a spatiotemporal semantic filtering mechanism using imaging parameters such as imaging time and shooting latitude. By modeling the imaging mechanism, the system can perform logical verification and consistency constraints on the detection results of the same target under different observation conditions. This not only improves the recognition accuracy of individual slices, but also ensures that the extraction of building increase and decrease information is more stable and reliable in long-term tasks such as urban renewal monitoring, effectively filtering out false change information caused by changes in illumination. Attached Figure Description

[0020] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 A flowchart illustrating a remote sensing image building detection method deeply coupled with remote sensing mechanisms according to an embodiment of this application is shown.

[0022] Figure 2 The schematic diagram illustrates a structural block diagram of a remote sensing image building detection device that is deeply coupled with remote sensing mechanisms according to an embodiment of this application;

[0023] Figure 3 A block diagram schematically illustrates an electronic device suitable for a remote sensing image building detection method with deep coupling of remote sensing mechanisms, according to an embodiment of this application. Detailed Implementation

[0024] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0028] Figure 1 A flowchart illustrating a remote sensing image building detection method deeply coupled with remote sensing mechanisms according to an embodiment of this application is shown.

[0029] like Figure 1 As shown, the remote sensing image building detection method with deep coupling of remote sensing mechanism in this embodiment includes steps S110 to S150.

[0030] In step S110, multiple remote sensing images containing the target building are acquired, along with the positioning information and imaging parameters of each remote sensing image.

[0031] For example, target buildings are man-made architectural structures that need to be automatically identified and extracted from remote sensing imagery. They typically have roofs, walls, and regular geometric shapes, distinguishing them from natural features such as roads and vegetation.

[0032] For example, multiple remote sensing images refer to two or more remote sensing images from different sources or with different imaging conditions. Different sources can be different satellite sensors, different imaging times, different shooting angles, etc.

[0033] For example, location information describes the position and orientation of remotely sensed images in geospatial space. It is used to convert influencing pixel coordinates into real geographic coordinates and mainly includes RPC coefficients (Rational Polynomial Coefficients), geometric positioning parameters, orbital parameters, geocoding data, etc.

[0034] For example, imaging parameters are data that record the sensor state and environmental conditions when remote sensing images are captured, including: pitch angle (the angle between the sensor pointing and the vertical direction), roll angle (the angle of rotation of the sensor around the flight axis), imaging time, and shooting latitude.

[0035] For example, acquiring multiple remote sensing images containing a target building refers to the process of reading or downloading the original image files and their associated metadata files from remote sensing data sources (such as satellite data providers, remote sensing data platforms, and local storage).

[0036] In step S120, geometric correction and coordinate unification are performed on multiple remote sensing images based on the positioning information to obtain a standardized remote sensing image set.

[0037] For example, positioning information can be RPC coefficients: a set of parameters that describe the mathematical relationship between image coordinates and ground coordinates (latitude, longitude, and elevation).

[0038] For example, positioning information can be geometric positioning parameters, including satellite orbit parameters, attitude parameters, imaging time, etc.

[0039] For example, geometric correction refers to the process of eliminating spatial location errors in remote sensing images caused by factors such as imaging system geometric distortion, sensor attitude changes, terrain undulations, and Earth curvature. It involves establishing a mathematical transformation relationship between image coordinates and ground coordinates to resample image pixels to the correct geographical location. Common correction models include polynomial correction models, affine transformation models, and projection transformation models.

[0040] For example, coordinate unification refers to transforming geometrically corrected source images from their original projection coordinate systems to the same preset map projection coordinate system (such as WGS84 / UTM), so that all images have the same spatial reference datum.

[0041] In step S130, the normalized remote sensing image set is sliced ​​according to the geographical range and resolution of the normalized remote sensing image set to obtain an initial image slice set.

[0042] For example, geographic extent refers to the geographical boundary of the area covered by a single standardized remote sensing image, which is usually represented by a rectangle.

[0043] Resolution refers to the actual ground size represented by each pixel in a remote sensing image, usually measured in meters per pixel.

[0044] For example, slicing is the process of cutting a large, standardized remote sensing image into several smaller sub-image blocks according to a preset size (in pixels) and step size.

[0045] In step S140, a sample index library is constructed based on the imaging parameters of each initial image slice in the initial image slice set. Imaging filtering rules are set based on the sample index library, and the target image slice set is selected from the initial image slice set according to the imaging filtering rules.

[0046] For example, a sample index library refers to a structured data storage structure used to record index entries for each image slice, enabling rapid retrieval of "image slice - imaging parameters - geographical location". The sample index library is used to establish the association between influencing slices and imaging parameters, supporting rapid retrieval and conditional queries.

[0047] For example, an imaging screening rule is a conditional expression based on imaging parameters, used to select image slices that meet specific imaging conditions from a sample index library.

[0048] In step S150, deep features of each target image slice are extracted from the target image slice set through a preset image backbone network. Imaging parameters and deep features are fused to obtain fused features. The category and location information of the target building are determined based on the fused features.

[0049] Image backbone network refers to a deep learning network model used to extract deep visual features from slices of target images.

[0050] Deep features can refer to high-dimensional semantic feature maps extracted from target image slices after nonlinear transformations such as multi-layer convolution, pooling, and activation of the image backbone network.

[0051] Feature fusion refers to the combination of deep visual features extracted by the image backbone network and auxiliary features encoded by imaging parameters to form a joint feature representation. Fusion methods include: concatenating features by connecting the beginning and end of two feature vectors, adding features element-wise when the dimensions are the same, and attention fusion with dynamic fusion of learned weights, etc.

[0052] The target building category is used to determine whether a pixel or region belongs to a building. The representation can be a binary representation of building / non-building or a multi-class representation such as residential / commercial / industrial.

[0053] The location information of the target building can determine the spatial position of the building in the image, for example, by using bounding box coordinates.

[0054] The remote sensing image building detection method based on the remote sensing mechanism deep coupling of the embodiments of this application introduces the imaging parameters of remote sensing images (such as pitch angle, roll angle, imaging time, shooting latitude, etc.) as physical constraints into the building detection process, realizes the joint modeling of image content and imaging geometry conditions, thereby significantly improving the accuracy and robustness of building detection. Specifically, geometric correction and coordinate unification based on positioning information eliminate spatial distortion and projection differences between multi-source images, providing a common geographic reference benchmark for subsequent processing. Secondly, through slicing and the construction of a sample index library based on imaging parameters, active control and screening of the imaging conditions of training samples are achieved, effectively avoiding the condition bias problem caused by the single imaging conditions and uneven spatial distribution of samples in traditional methods. Furthermore, the imaging parameters are fused with the deep visual features extracted by the backbone network, enabling the model to perceive and utilize the decisive influence of imaging geometry and lighting conditions on the appearance features of buildings (such as shadow length and side wall visibility), thereby achieving effective compensation for differences in the appearance of building targets and fundamentally solving the problem of model adaptability to multi-source remote sensing data. Finally, this method can effectively reduce the problems of missed detections, false detections, and boundary blurring caused by changes in imaging conditions, achieving high-precision and robust automatic detection of building categories and locations.

[0055] According to an embodiment of this application, obtaining the positioning information and imaging parameters of each remote sensing image includes: determining the positioning information and imaging parameters based on the metadata information attached to each remote sensing image; wherein, the imaging parameters include geographical location, imaging angle and image attributes.

[0056] For example, metadata refers to structured data that accompanies remote sensing image files and describes the inherent attributes and acquisition conditions of the images. Metadata typically exists as a separate file or embedded in the header of the image file.

[0057] For example, common metadata includes image identifiers, time information, georeferences (such as projected coordinate systems, RPC coefficients, and four-corner latitude and longitude), and sensor status (such as pitch and roll angles).

[0058] For example, location information refers to data extracted from metadata used to determine the mapping relationship between image pixels and geospatial locations. Location information is the foundation for achieving geometric correction and coordinate unification.

[0059] For example, imaging parameters refer to physical parameters extracted from metadata that describe the sensor state and environmental conditions during the remote sensing image acquisition process. Geographic location refers to the geospatial location information of the area at the time of image acquisition or the area covered by the image, such as the latitude at which it was captured. Imaging angle refers to the sensor's attitude angle and the angle of sunlight during imaging, such as pitch angle. / Roll angle Image attributes refer to the pixel-level features and data structure attributes of the image itself, such as imaging time. .

[0060] The remote sensing image building detection method based on the deep coupling of remote sensing mechanisms in this application realizes the unified acquisition and management of image spatial reference and imaging physical conditions by systematically extracting positioning information and imaging parameters from the metadata file attached to the remote sensing image. The positioning information provides a precise spatial mapping basis for subsequent geometric correction and coordinate unification, effectively eliminating geometric deformation and projection differences between multi-source images.

[0061] According to embodiments of this application, geometric correction and coordinate unification are performed on multiple remote sensing images based on positioning information to obtain a standardized remote sensing image set. This includes: georegistering multiple remote sensing images based on positioning information using a control point matching algorithm to construct a transformation relationship between each remote sensing image and a reference map projection; performing geometric correction on each remote sensing image based on the transformation relationship using a polynomial geometric correction model or an affine / projective transformation model; and projecting the geometrically corrected remote sensing images together onto a preset map projection coordinate system to obtain a standardized remote sensing image set with the same spatial reference.

[0062] For example, control point matching algorithms refer to algorithms that automatically or semi-automatically identify corresponding ground features (control points) between different images or between images and a reference map, and are used to establish spatial transformation relationships between images.

[0063] For example, georegistration refers to the process of aligning remote sensing images from different sources, at different times, and from different sensors to the same geographic coordinate reference system. By establishing the transformation relationship between image coordinates and ground coordinates, pixel-level alignment of multi-source images can be achieved in spatial location.

[0064] For example, geometric correction involves resampling an image by applying transformation relationships so that each pixel is mapped to the correct geographic location.

[0065] For example, projecting multiple geometrically corrected remote sensing images onto a preset map projection coordinate system refers to the mathematical transformation process of converting geographic coordinates (latitude, longitude) on the three-dimensional Earth ellipsoid into two-dimensional planar coordinates (eastward, northward). After projection transformation, all remote sensing images have a completely consistent coordinate reference system definition, including the same datum, projection type, projection parameters, and coordinate units.

[0066] For example, the pitch angle Roll angle Imaging time Latitude of shooting Together they form an imaging parameter feature vector: ,in, Indicates the dimension of imaging parameters. express A real vector space; establishing an index binding relationship between imaging parameters and normalized remote sensing images, forming the following data structure: ,in, Indicates the first A standardized remote sensing image, This is its corresponding imaging parameter vector.

[0067] For example, based on location information, multiple remote sensing images can be georeferenced using a control point matching algorithm to construct the transformation relationship between each remote sensing image and the reference map projection; then, using a polynomial geometric correction model or an affine / projective transformation model, geometric correction can be performed on each remote sensing image based on the transformation relationship, which can be achieved through the following steps:

[0068] (i) Based on the metadata information attached to each source remote sensing image, extract its image sensor model and georeferenced information, and extract the sensor model according to the rational function model (RPC) and imaging model formula:

[0069]

[0070] in, and These represent the row and column pixel coordinates in the remote sensing image, respectively. For geographic three-dimensional coordinates, This represents a polynomial function, with coefficients derived from RPC metadata of remote sensing imagery.

[0071] (ii) The mapping relationship between pixel space and geographic space between remote sensing images from different sources is obtained by fitting using the least squares method:

[0072]

[0073] Among them, the The pixel coordinates of the control points in the remote sensing image are Its corresponding map projection coordinates are , Represents the affine or projective transformation matrix.

[0074] (iii) Spatial resampling of the original remote sensing imagery is performed based on a polynomial geometric correction model to correct spatial distortions caused by factors such as viewpoint shift and terrain elevation differences:

[0075]

[0076]

[0077] in, These are the original pixel coordinates. Original pixel coordinate variables The power order is used to construct the polynomial correction term. The coordinates are after geometric correction. These are the fitting coefficients. The order of the polynomial ( ; )

[0078] The remote sensing image building detection method based on the remote sensing mechanism deep coupling of the embodiments of this application ensures high-precision alignment and superposition of multi-source images in spatial location, and provides a standardized spatial basis for subsequent slicing processing, imaging parameter association and feature fusion, thereby significantly reducing the building detection error caused by geometric deformation and projection inconsistency.

[0079] According to an embodiment of this application, a set of normalized remote sensing images is sliced ​​based on the geographic extent and resolution of the set to obtain an initial image slice set. This includes: obtaining the coordinate information, size information, and pixel information of each normalized remote sensing image from the geographic extent and resolution of the set; determining the geographic extent of each normalized remote sensing image based on the coordinate information, size information, and pixel information; and performing sliding window slicing on each normalized remote sensing image based on the geographic extent and pixel information to obtain the initial image slice set.

[0080] For example, coordinate information describes each remote sensing image. Data on location in geospatial space, including spatial projection coordinate system and geographic coordinates of the top left corner of the image. And the geographical coordinates of the four corners.

[0081] For example, size information refers to the pixel dimensions of a remote sensing image, including the image width. and image height .

[0082] For example, pixel information refers to the basic attribute information of remote sensing images, including the spatial resolution (pixel size) of the image. .

[0083] For example, based on the geographic coordinates of the top left corner of the image Image width Image height and pixel size The geographical extent of the remote sensing image can be calculated as follows: .

[0084] Sliding slicing refers to the operation of extracting sub-image blocks sequentially by sliding a fixed-size window (slice) across an image from left to right and from top to bottom according to a set step size. The sliding sequence is from left to right, and after completing a row, the step size is moved down, and the sliding continues from left to right until the entire image is covered.

[0085] For example, set the slice size to pixels, step size and ,in, and These are the pixel height and width of each slice, respectively. and These represent the sliding distances of adjacent slices in the horizontal and vertical directions, respectively.

[0086] For example, the initial set of image slices ,in, For the first A set of slices generated from a normalized remote sensing image. For the first The first standardized remote sensing image A slice, For the first The number of slices generated from a normalized remote sensing image. Also, each slice... Each region has a unique image region index and geographic location boundary, which can be determined by the pixel position at the top left corner of the slice window. With image geographic transformation matrix Calculate the geographical coordinates of its four corners.

[0087] The remote sensing image building detection method based on the remote sensing mechanism deep coupling of the embodiments of this application obtains the coordinate information, size information and pixel information of the standardized remote sensing images, accurately calculates the geographical range of each image, and performs sliding window tiling based on the geographical range and resolution to generate an initial image tile set with uniform pixel size and precise geographical location boundaries, laying a standardized data foundation for subsequent sample index library construction and deep learning feature extraction.

[0088] According to an embodiment of this application, a sample index library is constructed based on the imaging parameters of each initial image slice in the initial image slice set, including: generating a number for each initial image slice based on its location in a normalized remote sensing image; associating the number, geographical location, imaging angle, and image attributes of each initial image slice to form an index entry for the initial image slice; and forming a sample index library based on multiple index entries of the initial image slice set.

[0089] For example, a number is a unique identifier assigned to each initial image slice, used for quick retrieval and location of the slice in the sample index. The number can be combined with the original image identifier and the slice's row and column names.

[0090] For example, an index entry refers to a record in the sample index database that contains all the associated information for a single slice. Each index entry is a structured data object.

[0091] For example, for each slice The following index entries were generated:

[0092]

[0093] in, For slices The corresponding index record, For slices Unique number, For slices The geographical coordinates of the top left corner or center point For containing slices The set of four corner coordinates of the boundary , For the first Imaging parameters of a standardized remote sensing image and , For the first A unique identifier for a standardized remote sensing image.

[0094] For example, each of the above slices The index entries are organized into a queryable data set, forming a sample index library. .

[0095] The remote sensing image building detection method based on the deep coupling of remote sensing mechanisms in this application realizes precise control and scientific screening of training sample distribution by constructing a joint index system of imaging parameters and image content. In this way, the model can learn the invariant characteristics of buildings under different physical parameters, which greatly enhances the robustness of the detection algorithm in different geographical regions and different seasonal phases, and enables the system to have stronger scene transfer capabilities.

[0096] According to an embodiment of this application, selecting a target image slice set from an initial image slice set according to imaging screening rules includes: reading the geographical location, imaging angle, and image attributes of each initial image slice from a sample index library; determining whether the geographical location, imaging angle, and image attributes of each initial image slice all conform to the imaging screening rules; and taking each initial image slice that conforms to the imaging screening rules as a target image slice to form a target image slice set.

[0097] For example, imaging screening rules are based on pre-defined conditional expressions of imaging parameters, used to select image slices from a sample index that meet specific imaging conditions. Imaging screening rules include, but are not limited to, imaging time intervals. Roll angle or pitch angle threshold and latitude range .

[0098] For example, the target image slice meets specific spatiotemporal conditions, that is, it meets the condition constraints specified by the user or system, which have a clear spatial range and / or temporal range.

[0099] The remote sensing image building detection method based on the remote sensing mechanism deeply coupled with embodiments of this application constructs a spatiotemporal semantic filtering mechanism using parameters such as imaging time and shooting latitude. By modeling the imaging mechanism, the system can perform logical verification and consistency constraints on the detection results of the same target under different observation conditions. This not only improves the recognition accuracy of individual slices but also ensures that the extraction of building increase and decrease information is more stable and reliable in long-term tasks such as urban renewal monitoring, effectively filtering out false change information caused by changes in illumination.

[0100] According to an embodiment of this application, the method further includes: after extracting the deep features of each target image slice from the target image slice set through a preset image backbone network, enhancing the deep features through a preset texture edge enhancement module to obtain enhanced deep features, and replacing the extracted deep features with the enhanced deep features.

[0101] For example, an image backbone network refers to a deep learning network model used to extract deep visual features from slices of target images, serving as a feature extractor for the entire detection process.

[0102] For example, deep features refer to high-dimensional semantic feature maps extracted from target image slices after multi-layer convolution, pooling, activation and other non-linear transformations through the image backbone network.

[0103] For example, slice each target image The image is input into a convolutional neural network on the backbone to extract deep features. express:

[0104]

[0105] in, This represents the mapping function of a convolutional neural network. For network functions, These represent the number of channels, height, and width of the output deep feature map, respectively. It represents a three-dimensional real tensor space.

[0106] For example, after obtaining deep features, a specially designed texture edge enhancement module can be introduced. By combining edge detection operators and texture filters, deeper features are further enhanced:

[0107]

[0108] in, This is the feature output after processing by the texture edge enhancement module. The enhanced feature is then used. Replace the original deep features This serves as the input for subsequent fusion and detection.

[0109] Edge detection operators are used to extract geometric information such as building outlines and roof boundaries; texture filters are used to capture repetitive local features such as roof material and surface patterns.

[0110] The remote sensing image building detection method based on the remote sensing mechanism deep coupling of the present application embodiment inputs the deep features extracted by the backbone neural network into the enhancement module that combines edge detection operator and texture filter, outputs features with the same dimension but significantly enhanced edge and texture information, and uses them to replace the original deep features, so as to enhance the building outline localization and roof texture recognition capabilities and improve the subsequent detection accuracy.

[0111] According to an embodiment of this application, imaging parameters and deep features are fused to obtain fused features, and the category and location information of the target building are determined based on the fused features. The method includes: mapping imaging parameters into auxiliary feature vectors through a preset multilayer perceptron; fusing deep features and auxiliary feature vectors to obtain fused features; and performing convolution and fully connected processing on the fused features in sequence to obtain the category and location information of the target building.

[0112] A multi-layer perceptron (MLP) is a feedforward neural network consisting of an input layer, several hidden layers, and an output layer, used to map imaging parameters into high-dimensional auxiliary feature vectors.

[0113] For example, it will contain the first The first standardized remote sensing image Pitch angle in each slice Roll angle Imaging time and shooting latitude Imaging parameters The input is fed into a multilayer perceptron (MLP), where, through several connection layers and activation functions, the imaging parameters are mapped into auxiliary feature vectors compatible with the dimensions of the image feature space. The following formula can be used to calculate it:

[0114]

[0115] in, For network functions, Represents auxiliary feature vectors It is a three-dimensional tensor with dimensions of .

[0116] Through repetition or broadcasting mechanisms, auxiliary feature vectors can be matched with deep image features in spatial dimensions.

[0117] For example, the deep features of the image output by the image backbone network With auxiliary feature vectors The two concatenate at the channel dimension and input them into subsequent convolutional layers to obtain the fused feature. The following formula can be used to calculate it:

[0118]

[0119] in, Indicates channel splicing. To merge convolutional layer functions, This is the activation function.

[0120] For example, fusion features Convolutional operations are performed to further extract spatial features and semantic information. Fully connected processing involves flattening the feature map after convolution and mapping it to the final output space through fully connected layers. The final outputs are classification probabilities (classes) and bounding box coordinates (location information). The classification probability is used to determine whether it is a building, and the bounding box coordinates are used to output the building's location.

[0121] The remote sensing image building detection method based on the remote sensing mechanism deep coupling of embodiments of this application, by setting an independent physical parameter input branch in the deep neural network, nonlinearly fuses the imaging parameters of satellite imagery (including but not limited to elevation angle, azimuth angle, solar altitude angle, etc.) with pixel-level deep features. Using physical parameters as prior constraints, the model is guided to automatically perceive changes in imaging geometry and illumination conditions during the feature extraction stage, thereby achieving effective compensation for differences in the appearance of building targets and fundamentally solving the problem of model adaptability to multi-source remote sensing data.

[0122] In the embodiments of this application, deep features are first extracted from remote sensing image slices using an encoder (such as an Encoder). Simultaneously, the imaging parameter vector is input into an MLP encoder and mapped to an auxiliary feature vector. The auxiliary feature vector is then adjusted to match the size of the encoded feature vector through broadcast expansion. Afterward, the extracted deep features and the auxiliary feature vector are fed into a feature fusion module for feature fusion. The fused features are then input into a decoder (such as a Decoder) to restore the resolution through upsampling and skip connections. Finally, the building target recognition result is output by the detection head.

[0123] Based on the aforementioned remote sensing mechanism-deeply coupled remote sensing image building detection method, this application also provides a remote sensing mechanism-deeply coupled remote sensing image building detection device, which is described below in conjunction with... Figure 2 The device is described in detail.

[0124] Figure 2 A schematic diagram of a remote sensing image building detection device with deep coupling of remote sensing mechanism according to an embodiment of this application is shown.

[0125] like Figure 2As shown, the remote sensing image building detection device 200 with deep coupling of remote sensing mechanism in this embodiment includes a parameter acquisition module 210, an image correction module 220, an image slicing module 230, a slice selection module 240, and an information determination module 250.

[0126] The parameter acquisition module 210 is used to acquire multiple remote sensing images containing the target building, as well as the positioning information and imaging parameters of each remote sensing image. In one embodiment, the parameter acquisition module 210 can be used to perform step S110 described above, which will not be repeated here.

[0127] The image correction module 220 is used to perform geometric correction and coordinate unification on multiple remote sensing images based on positioning information to obtain a standardized remote sensing image set. In one embodiment, the image correction module 220 can be used to perform step S120 described above, which will not be repeated here.

[0128] The image slicing module 230 is used to slice the normalized remote sensing image set according to its geographical extent and resolution to obtain an initial image slice set. In one embodiment, the image slicing module 230 can be used to perform step S130 described above, which will not be repeated here.

[0129] The slice selection module 240 is used to construct a sample index library based on the imaging parameters of each initial image slice in the initial image slice set, set imaging selection rules based on the sample index library, and select the target image slice set from the initial image slice set according to the imaging selection rules. In one embodiment, the slice selection module 240 can be used to perform step S140 described above, which will not be repeated here.

[0130] The information determination module 250 is used to extract deep features of each target image slice from the target image slice set through a preset image backbone network, fuse imaging parameters with deep features to obtain fused features, and determine the category and location information of the target building based on the fused features. In one embodiment, the information determination module 250 can be used to perform step S150 described above, which will not be repeated here.

[0131] According to embodiments of this application, any multiple modules among the parameter acquisition module 210, image correction module 220, image slicing module 230, slice selection module 240, and information determination module 250 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the parameter acquisition module 210, image correction module 220, image slicing module 230, slice selection module 240, and information determination module 250 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPMA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the parameter acquisition module 210, image correction module 220, image slicing module 230, slice selection module 240, and information determination module 250 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0132] Figure 3 A block diagram schematically illustrates an electronic device suitable for a remote sensing image building detection method with deep coupling of remote sensing mechanisms, according to an embodiment of this application.

[0133] like Figure 3 As shown, an electronic device 300 according to an embodiment of this application includes a processor 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage portion 304 into a random access memory (RAM) 303. The processor 301 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 301 may also include onboard memory for caching purposes. The processor 301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0134] RAM 303 stores various programs and data required for the operation of electronic device 300. Processor 301, ROM 302, and RAM 303 are interconnected via bus 304. Processor 301 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 302 and / or RAM 303. It should be noted that the programs may also be stored in one or more memories other than ROM 302 and RAM 303. Processor 301 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0135] According to embodiments of this application, the electronic device 300 may further include an input / output (I / O) interface 305, which is also connected to a bus 304. The electronic device 300 may also include one or more of the following components connected to the input / output (I / O) interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the input / output (I / O) interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 310 as needed so that computer programs read from it can be installed into the storage section 308 as needed.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0137] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined or combined in various ways without departing from the spirit and teachings of this application. All such combinations or combinations fall within the scope of this application.

[0138] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for building detection in remote sensing images based on deep coupling of remote sensing mechanisms, characterized in that, include: Acquire multiple remote sensing images containing the target building, as well as the location information and imaging parameters of each remote sensing image; Based on the positioning information, geometric correction and coordinate unification are performed on the multiple remote sensing images to obtain a standardized remote sensing image set. The standardized remote sensing image set is sliced ​​according to its geographical range and resolution to obtain an initial image slice set. A sample index library is constructed based on the imaging parameters of each initial image slice in the initial image slice set. Imaging filtering rules are set based on the sample index library, and a target image slice set is selected from the initial image slice set according to the imaging filtering rules. The deep features of each target image slice are extracted from the target image slice set by a preset image backbone network. The imaging parameters and the deep features are fused to obtain fused features. The category and location information of the target building are determined based on the fused features.

2. The method according to claim 1, characterized in that, Obtaining the positioning information and imaging parameters of each of the remote sensing images includes: The positioning information and imaging parameters are determined based on the metadata information attached to each of the remote sensing images; wherein, the imaging parameters include geographical location, imaging angle and image attributes.

3. The method according to claim 1, characterized in that, The process of performing geometric correction and coordinate unification on the multiple remote sensing images based on the positioning information to obtain a standardized remote sensing image set includes: Based on the positioning information, the multiple remote sensing images are georegistered using a control point matching algorithm to construct the transformation relationship between each remote sensing image and the reference map projection. A polynomial geometric correction model or an affine / projective transformation model is used to perform geometric correction on each of the remote sensing images based on the transformation relationship; Multiple remote sensing images, after geometric correction, are projected together onto a preset map projection coordinate system to obtain a standardized set of remote sensing images with the same spatial reference.

4. The method according to claim 1, characterized in that, The step involves slicing the standardized remote sensing image set according to its geographical extent and resolution to obtain an initial image slice set, including: The coordinate information, size information, and pixel information of each normalized remote sensing image are obtained from the geographic range and resolution of the normalized remote sensing image set. Based on the coordinate, size, and pixel information of each standardized remote sensing image, the geographical extent of each standardized remote sensing image is determined. Based on the geographical extent and pixel information of each standardized remote sensing image, a sliding window slicing is performed on each standardized remote sensing image to obtain the initial image slice set.

5. The method according to claim 2, characterized in that, The step of constructing a sample index library based on the imaging parameters of each initial image slice in the initial image slice set includes: Each initial image slice is assigned a number based on its position in the normalized remote sensing image. The initial image slice is associated with its number, geographical location, imaging angle, and image attributes to form an index entry for the initial image slice; The sample index library is formed based on multiple index entries of the initial image slice set.

6. The method according to claim 5, characterized in that, The step of selecting the target image slice set from the initial image slice set according to the imaging screening rules includes: The geographical location, imaging angle, and image attributes of each initial image slice are read from the sample index library; Determine whether the geographical location, imaging angle, and image attributes of each initial image slice conform to the imaging screening rules; Each initial image slice that meets the imaging screening rules is taken as a target image slice, forming the target image slice set.

7. The method according to claim 1, characterized in that, The method further includes: After extracting the deep features of each target image slice from the target image slice set using a preset image backbone network, The deep features are enhanced by a preset texture edge enhancement module to obtain enhanced deep features, which then replace the extracted deep features.

8. The method according to claim 1, characterized in that, The step of fusing the imaging parameters with the deep features to obtain fused features, and determining the category and location information of the target building based on the fused features, includes: The imaging parameters are mapped into auxiliary feature vectors using a pre-defined multilayer perceptron. The deep features are fused with the auxiliary feature vector to obtain the fused features; The fused features are sequentially processed by convolution and fully connected layers to obtain the category and location information of the target building.

9. A remote sensing image building detection device with deep coupling of remote sensing mechanism, characterized in that, include: The parameter acquisition module is used to acquire multiple remote sensing images containing the target building, as well as the positioning information and imaging parameters of each remote sensing image; The image correction module is used to perform geometric correction and coordinate unification on the multiple remote sensing images based on the positioning information to obtain a standardized remote sensing image set. The image slicing module is used to slice the standardized remote sensing image set according to the geographical range and resolution of the standardized remote sensing image set to obtain an initial image slice set. The slice filtering module is used to construct a sample index library based on the imaging parameters of each initial image slice in the initial image slice set, set imaging filtering rules based on the sample index library, and filter out the target image slice set from the initial image slice set according to the imaging filtering rules. The information determination module is used to extract deep features of each target image slice from the target image slice set through a preset image backbone network, fuse the imaging parameters with the deep features to obtain fused features, and determine the category and location information of the target building based on the fused features.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images