Building matching method and device based on remote sensing image, electronic equipment and medium

Abstract

The embodiment of the invention discloses a building matching method and device based on a remote sensing image, electronic equipment and a medium. A specific embodiment of the method comprises the following steps: acquiring a remote sensing image shot for a target building area and building reference information corresponding to the target building area; performing image preprocessing on the remote sensing image to generate a preprocessed remote sensing image; performing building segmentation for the preprocessed remote sensing image by using a pre-trained building matching model to obtain a building segmentation image; extracting an object feature information set corresponding to each building segmentation object included in the building segmentation image; according to the object feature information set and the building reference information, building matching for the remote sensing image is executed to generate a building matching result; and carrying out visualization processing on the building matching result. According to the embodiment, building matching for the remote sensing image can be accurately and efficiently realized.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, and medium for building matching based on remote sensing images. Background Art

[0002] At present, with the continuous development of remote sensing technology, various studies on remote sensing images have become the main development direction. Building matching in remote sensing images has also become the current mainstream research direction. For building matching in remote sensing images, the usual method is to achieve the matching of buildings in remote sensing images through manual comparison.

[0003] However, when using the above method to match buildings in remote sensing images, the following technical problems often occur: The manual matching method is not only arduous, but also often results in matching errors, leading to low matching efficiency.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a building matching method, device, electronic device and medium based on remote sensing images to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a building matching method, comprising: acquiring a remote sensing image taken for a target building area and building reference information corresponding to the target building area; performing image preprocessing on the remote sensing image to generate a preprocessed remote sensing image; using a pre-trained building matching model to perform building segmentation on the preprocessed remote sensing image to obtain a building segmentation image; extracting an object feature information set corresponding to each building segmentation object included in the building segmentation image; performing building matching on the remote sensing image based on the object feature information set and the building reference information to generate a building matching result; and visualizing the building matching result.

[0008] In a second aspect, some embodiments of the present disclosure provide a building matching device based on remote sensing images, comprising: an acquisition unit, configured to acquire a remote sensing image taken for a target building area and building reference information corresponding to the target building area; a preprocessing unit, configured to perform image preprocessing on the remote sensing image to generate a preprocessed remote sensing image; a segmentation unit, configured to use a pre-trained building matching model to perform building segmentation on the preprocessed remote sensing image to obtain a building segmentation image; an extraction unit, configured to extract an object feature information set corresponding to each building segmentation object included in the building segmentation image; an execution unit, configured to perform building matching on the remote sensing image according to the object feature information set and the building reference information to generate a building matching result; and a visualization unit, configured to visualize the building matching result.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0011] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the building matching method of some embodiments of the present disclosure, the building matching for remote sensing images can be accurately and efficiently achieved. Specifically, the reason why the building matching of the relevant remote sensing images is not accurate and efficient is that the manual comparison method is not only arduous, but also often has matching errors, resulting in low matching efficiency. Based on this, the building matching method of some embodiments of the present disclosure first obtains the remote sensing image taken for the target building area and the building reference information corresponding to the above target building area. Here, the acquired remote sensing image is used as an image data source for building matching. The building reference information can be used as a basic data set for subsequent building matching, which is convenient for subsequent accurate matching. Then, the above remote sensing image is image preprocessed to generate a preprocessed remote sensing image to remove image noise in the remote sensing image and ensure the accuracy of subsequent building matching. Then, using the pre-trained building matching model, the building segmentation for the above preprocessed remote sensing image can be accurately and quickly performed to obtain a building segmentation image. Then, the object feature information set corresponding to each building segmentation object included in the above building segmentation image can be accurately extracted to facilitate the subsequent building matching. Furthermore, based on the above object feature information set and the above building reference information, the building matching for the above remote sensing image can be accurately performed to generate a building matching result. Finally, the above building matching result is visualized to display the matching result. In summary, based on the building matching model, the building segmentation image can be accurately and efficiently generated. On this basis, by accurately generating the object feature information set corresponding to the building segmentation image, accurate matching with the building reference information can be achieved, and efficient matching of the building can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flow chart of some embodiments of the building matching method according to the present disclosure; Figure 2 is a schematic diagram of the structure of some embodiments of the building matching device according to the present disclosure; Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0020] refer to Figure 1 , shows a process 100 of some embodiments of the building matching method according to the present disclosure. The building matching method comprises the following steps: Step 101: Acquire a remote sensing image taken for a target building area and building reference information corresponding to the target building area.

[0021] In some embodiments, the execution subject (e.g., electronic device) of the above-mentioned building matching method can obtain the remote sensing image taken for the target building area and the building reference information corresponding to the above-mentioned target building area through a wired connection method or a wireless connection method. Among them, the target building area can be a building area photographed by a remote sensing shooting device. The building area can include buildings of various styles. For example, the target building area can be Haidian District, Beijing. The buildings corresponding to the target building area can be various buildings in Haidian District, Beijing. The remote sensing image can be an image taken by the remote sensing shooting device toward the target building area. The building reference information can be a real reference data set pre-stored for each building in the target building area. In practice, the building reference information can include: vector contour data corresponding to each building, and elevation data corresponding to each building. In practice, the building reference information can be a plurality of building key-value pairs formed with each building in the target building area. The key in the building key-value pair is the building identifier, and the pair in the building key-value pair is the reference information corresponding to the building. Multiple building key-value pairs can be stored in the target database.

[0022] In some optional implementations of some embodiments, after step 101, the steps further include: The first step is to obtain the regional road distribution information corresponding to the target building area. The regional road distribution information can characterize the road distribution of each road in the target building area. In practice, the regional road distribution information can be a road distribution map with road contours corresponding to each road in the target building area. The regional road distribution information can characterize the distribution of each road in the regional map corresponding to the target building area. In practice, the regional road distribution information can be a distribution map of the target building area at a predetermined map scale after the roads are color-characterized.

[0023] The second step is to extract the road distribution feature information corresponding to the remote sensing image. The road distribution feature information can represent the feature content corresponding to the distribution features of each road in the remote sensing image. In practice, the distribution features may include: road number features, road curvature features, road intersection features, and road direction features.

[0024] As an example, the execution subject may input the remote sensing image into a pre-trained road distribution feature extraction model to generate road distribution feature information. The road distribution feature extraction model may include multiple convolutional layers, each of which outputs road distribution feature information under corresponding features.

[0025] The third step is to crop the remote sensing image according to the regional road distribution information and the road distribution feature information to obtain a cropped image.

[0026] As an example, first, at least one regional road distribution sub-information matching the similarity between the distribution information corresponding to each distribution feature in the road distribution feature information is screened out from the regional road distribution information. Then, the distribution vector corresponding to each regional road distribution sub-information is determined, and the distribution feature vector corresponding to the road distribution feature information is determined as the first distribution vector and the second distribution vector, respectively. Next, the regional road distribution sub-information corresponding to the first distribution vector and the second distribution vector having the greatest similarity is screened out from at least one regional road distribution sub-information as the target regional road distribution sub-information. Finally, the remote sensing image is cropped according to the distribution area corresponding to the target regional road distribution sub-information to obtain a cropped image.

[0027] The fourth step is to determine the above cropped image as a remote sensing image.

[0028] In some optional implementations of some embodiments, the step of cropping the remote sensing image according to the regional road distribution information and the road distribution feature information to obtain the cropped image may include the following steps: The first step is to use the regional road key feature point generation model to generate the first road key feature point information set corresponding to the regional road distribution information. The regional road key feature point generation model can be a neural network model for generating road key feature point information. The road key feature point can be road intersection feature information. The road key feature point information can be feature information corresponding to the road intersection. The road key feature information can include: direction information corresponding to the road intersection, intersection information corresponding to the road intersection, and road width information corresponding to the road intersection. In practice, the regional road key feature point generation model can be a multi-classification model based on a convolutional layer. The regional road distribution information includes: a road intersection set. There is a one-to-one correspondence between the road intersections in the road intersection set and the first road key features in the first road key feature point information set.

[0029] The second step is to use the regional road key feature point generation model to generate a second road key feature point information set corresponding to the road distribution feature information, wherein the road intersections in the road intersection set corresponding to the road distribution feature information have a one-to-one correspondence with the second road key feature point information in the second road key feature point information set.

[0030] The third step is to compare the feature point information of the first road key feature point information set with the second road key feature point information set to generate a comparison result. The comparison result may be at least one comparison information representing the similarity of corresponding features between each road intersection. The feature similarity may represent the degree of similarity of feature information between road intersections.

[0031] The fourth step is to crop the remote sensing image according to the comparison result to obtain a cropped image.

[0032] As an example, first, the execution subject may use the comparison information in which the similarity of each road intersection in the comparison result is greater than the preset similarity as the target comparison information, and then crop the remote sensing image according to the target comparison information to obtain a cropped image.

[0033] Step 102: performing image preprocessing on the remote sensing image to generate a preprocessed remote sensing image.

[0034] In some embodiments, the execution entity may perform image preprocessing on the remote sensing image to generate a preprocessed remote sensing image.

[0035] In some optional implementations of some embodiments, the above-mentioned image preprocessing of the remote sensing image to generate a preprocessed remote sensing image may include the following steps: The first step is to perform radiation correction on the remote sensing image to generate a radiation corrected image.

[0036] The second step is to perform atmospheric correction on the above-mentioned radiation correction image to generate an atmospheric correction image.

[0037] The third step is to perform geometric correction on the above-mentioned atmospheric correction image to generate a geometric correction image.

[0038] The fourth step is to generate the above-mentioned preprocessed remote sensing image based on the above-mentioned geometric correction image.

[0039] As an example, the execution entity may determine the geometrically corrected image as the pre-processed remote sensing image.

[0040] Step 103, using the pre-trained building matching model, execute building segmentation on the pre-processed remote sensing image to obtain a building segmentation image.

[0041] In some embodiments, the execution subject may use a pre-trained building matching model to perform building segmentation on the pre-processed remote sensing image to obtain a building segmentation image. The building matching model may be a neural network model for building matching based on image recognition technology. For example, the building matching model may be a YOLO model.

[0042] In some optional implementations of some embodiments, the above-mentioned use of the pre-trained building matching model to perform building segmentation on the above-mentioned pre-processed remote sensing image to obtain the building segmentation image may include the following steps: In the first step, the pre-processed remote sensing image is input into a plurality of first convolution layers connected in series in the building matching model to generate first convolution information. In practice, the first convolution information may be information in a vector form.

[0043] In the second step, a maximum pooling process of a predetermined specification is performed on the first convolution information to generate first pooling information.

[0044] The third step is to input the first pooling information into a plurality of second convolutional layers connected in series included in the building matching model to generate second convolutional information.

[0045] In the fourth step, the second convolution information is subjected to a maximum pooling process of a predetermined specification to generate second pooling information, wherein the predetermined specification may be a pre-set pooling specification.

[0046] In the fifth step, the second pooling information is input into a plurality of third convolutional layers connected in series included in the building matching model to generate third convolutional information.

[0047] In the sixth step, a maximum pooling process of a predetermined specification is performed on the third convolution information to generate third pooling information.

[0048] In the seventh step, the third pooling information is input into a plurality of fourth convolutional layers connected in series included in the building matching model to generate fourth convolutional information.

[0049] In the eighth step, a maximum pooling process of a predetermined specification is performed on the fourth convolution information to generate fourth pooling information.

[0050] In the ninth step, the fourth pooling information is input into a plurality of fifth convolutional layers connected in series included in the building matching model to generate fifth convolutional information.

[0051] In the tenth step, up-sampling is performed on the fifth convolution information to generate first up-sampling information.

[0052] In the eleventh step, an average pooling process of a predetermined specification is performed on the third convolution information to generate fifth pooling information.

[0053] In the twelfth step, the fifth pooling information is input into a plurality of sixth convolutional layers connected in series included in the building matching model to generate sixth convolutional information.

[0054] In the thirteenth step, first splicing information is generated according to the sixth convolution information, the first up-sampling information and the fourth convolution information.

[0055] As an example, the execution entity may perform vector concatenation on a vector corresponding to the sixth convolution information, a vector corresponding to the first up-sampling information, and a vector corresponding to the fourth convolution information to generate first concatenation information.

[0056] In step 14, up-sampling is performed on the first splicing information to generate second up-sampling information.

[0057] In the fifteenth step, an average pooling process of a predetermined specification is performed on the second convolution information to generate sixth pooling information.

[0058] In the sixteenth step, the sixth pooling information is input into a plurality of seventh convolutional layers connected in series included in the building matching model to generate seventh convolutional information.

[0059] Step 17: Generate second splicing information according to the seventh convolution information, the second up-sampling information and the third convolution information.

[0060] In step 18, up-sampling is performed on the second splicing information to generate third up-sampling information.

[0061] In the nineteenth step, an average pooling process of a predetermined specification is performed on the first convolution information to generate seventh pooling information.

[0062] In the twentieth step, the seventh pooling information is input into the multiple eighth convolutional layers connected in series included in the building matching model to generate eighth convolutional information; and third splicing information is generated according to the eighth convolutional information, the third upsampling information and the second convolutional information.

[0063] In the twenty-first step, up-sampling is performed on the third splicing information to generate fourth up-sampling information.

[0064] Step 22: Generate fourth splicing information according to the fourth up-sampling information and the first convolution information.

[0065] In step 23, the fourth splicing information is input into the plurality of serially connected ninth convolutional layers included in the building matching model to generate a building segmentation image.

[0066] Optionally, the generating the first splicing information according to the sixth convolution information, the first upsampling information and the fourth convolution information may include the following steps: In the first step, the fourth convolution information is input into a convolutional neural network based on an attention mechanism to generate the first attention information.

[0067] The second step is to input the sixth convolution information into the convolutional neural network based on the attention mechanism to generate the second attention information.

[0068] The third step is to splice the first attention information, the first upsampling information and the second attention information to generate the first splicing information.

[0069] In some optional implementations of some embodiments, the above-mentioned use of the pre-trained building matching model to perform building segmentation on the above-mentioned pre-processed remote sensing image to obtain the building segmentation image may include the following steps: The first step is to input the preprocessed remote sensing image into the plurality of serially connected tenth convolutional layers included in the building matching model to generate tenth convolution information.

[0070] In the second step, a maximum pooling process of a predetermined specification is performed on the tenth convolution information to generate eighth pooling information.

[0071] The third step is to input the eighth pooling information into the multiple serially connected eleventh convolutional layers included in the building matching model to generate the eleventh convolutional information.

[0072] In the fourth step, the eleventh convolution information is subjected to a maximum pooling process of a predetermined specification to generate ninth pooling information.

[0073] In the fifth step, the ninth pooling information and the eighth pooling information are pooled and fused to generate first fused information.

[0074] In the sixth step, the first fusion information is input into the plurality of serially connected twelfth convolutional layers included in the building matching model to generate twelfth convolutional information.

[0075] In the seventh step, a maximum pooling process of a predetermined specification is performed on the twelfth convolution information to generate tenth pooling information.

[0076] In the eighth step, the tenth pooling information and the ninth pooling information are pooled and fused to generate second fused information.

[0077] In the ninth step, the second fusion information is input into the plurality of serially connected thirteenth convolutional layers included in the building matching model to generate the thirteenth convolutional information.

[0078] In the tenth step, the thirteenth convolution information is subjected to a maximum pooling process of a predetermined specification to generate an eleventh pooling information.

[0079] In the eleventh step, the eleventh pooling information is fused with the tenth pooling information to generate third fused information.

[0080] In the twelfth step, the third fusion information is input into the multiple series-connected fourteenth convolutional layers included in the building matching model to generate fourteenth convolutional information.

[0081] In the thirteenth step, up-sampling is performed on the fifteenth convolution information to generate fifth up-sampling information.

[0082] In the fourteenth step, an average pooling process of a predetermined specification is performed on the second fusion information to generate twelfth pooling information.

[0083] In the fifteenth step, the twelfth pooling information is input into the multiple series-connected fifteenth convolutional layers included in the building matching model to generate fifteenth convolutional information.

[0084] In the sixteenth step, the fifth up-sampling information, the fifteenth convolution information and the thirteenth convolution information are concatenated to generate fifth concatenated information.

[0085] Step 17: up-sample the fifth spliced ​​information to generate sixth up-sampled information.

[0086] In the eighteenth step, an average pooling process of a predetermined specification is performed on the first fusion information to generate a thirteenth pooling information.

[0087] In the nineteenth step, the thirteenth pooling information is input into the multiple series-connected sixteenth convolutional layers included in the building matching model to generate sixteenth convolutional information.

[0088] In the twentieth step, the sixth up-sampling information, the sixteenth convolution information and the twelfth convolution information are concatenated to generate sixth concatenated information.

[0089] In the 21st step, up-sampling is performed on the sixth splicing information to generate seventh up-sampling information.

[0090] In the twenty-second step, an average pooling process of a predetermined specification is performed on the tenth convolution information to generate fourteenth pooling information.

[0091] In the twenty-third step, the fourteenth pooling information is input into the multiple series-connected seventeenth convolutional layers included in the building matching model to generate the seventeenth convolutional information.

[0092] In step 24, the seventh up-sampling information, the seventeenth convolution information and the eleventh convolution information are concatenated to generate seventh concatenated information.

[0093] In step 25, up-sampling is performed on the seventh splicing information to generate eighth up-sampling information.

[0094] In step 26, the eighth up-sampling information and the tenth convolution information are concatenated to generate eighth concatenated information.

[0095] In step 27, the eighth splicing information is input into the multiple series-connected eighteenth convolutional layers included in the building matching model to generate a building segmentation image.

[0096] Step 104: extracting an object feature information set corresponding to each building segmentation object included in the building segmentation image.

[0097] In some embodiments, the execution subject may extract a set of object feature information corresponding to each building segmentation object included in the building segmentation image, wherein the object feature information may be feature semantic information corresponding to each object feature of the building segmentation object.

[0098] As an example, the execution subject may input the sub-image corresponding to each building segmentation object into the object feature information generation model to generate object feature information and obtain an object feature information set. The object feature information generation model may be a convolutional layer with multiple layers connected in series.

[0099] Step 105: Perform building matching on the remote sensing image according to the object feature information set and the building reference information to generate a building matching result.

[0100] In some embodiments, the execution subject may perform building matching for the remote sensing image according to the object feature information set and the building reference information to generate a building matching result. The building reference information includes: each building reference sub-information corresponding to each building.

[0101] As an example, first, the execution subject may determine the object feature information and building reference information corresponding to each building object in the remote sensing image as the target feature information and the target building reference information, and obtain the target object feature information set and the target building reference sub-information set. Then, based on the object feature information set, the building reference information, the target object feature information set and the target building reference sub-information set, filter out the building object group whose similarity between the corresponding object feature information is greater than the similarity between the target object feature information, and whose similarity between the building reference sub-information is greater than the similarity between the target building reference information, and obtain at least one building object group. The similarity between the object feature information may be the object feature information similarity between the object feature information and the target object feature information. The target building reference information similarity may be the information similarity between the building reference sub-information and the target building reference sub-information. Finally, generate a building matching result for at least one building object group.

[0102] In some optional implementations of some embodiments, performing building matching for the remote sensing image according to the object feature information set and the building reference information to generate a building matching result includes the following steps: In the first step, for each building benchmark sub-information in the building benchmark information, the following first information generation step is performed: Sub-step 1: determine the object feature information corresponding to the building reference sub-information and the first road key feature point information corresponding to the adjacent relationship as the first object feature information and the first target road key feature point information.

[0103] Sub-step 2: for each remaining building benchmark sub-information in the remaining building benchmark sub-information set, perform the following second information generation step: The first sub-step is to determine the second object feature information and the second target road key feature point information corresponding to the remaining building benchmark sub-information.

[0104] The second sub-step is to determine the building benchmark information similarity between the remaining building benchmark sub-information and the building benchmark sub-information, determine the object feature information similarity between the first object feature information and the second object feature information, and determine the key feature point similarity between the first target road key feature point information and the second target road key feature point information.

[0105] The third sub-step is to perform weighted summation processing on the building benchmark information similarity, the object feature information similarity and the key feature point similarity to obtain the weighted sum similarity, wherein the weight corresponding to the building benchmark information similarity, the weight corresponding to the object feature information similarity and the weight corresponding to the key feature point similarity can be preset.

[0106] Sub-step 3: Select the remaining building benchmark sub-information with the largest corresponding weighted sum similarity from the remaining building benchmark sub-information set as the target remaining building benchmark sub-information.

[0107] Sub-step 4: generating building sub-matching information between the building information corresponding to the target remaining building benchmark sub-information and the building information corresponding to the building benchmark sub-information.

[0108] In the second step, the obtained sub-matching information of each building is determined as the building matching result.

[0109] Step 106: Visualize the building matching results.

[0110] In some embodiments, the execution entity may perform visualization processing on the building matching results.

[0111] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the building matching method of some embodiments of the present disclosure, the building matching for remote sensing images can be accurately and efficiently achieved. Specifically, the reason why the building matching of the relevant remote sensing images is not accurate and efficient is that the manual comparison method is not only arduous, but also often has matching errors, resulting in low matching efficiency. Based on this, the building matching method of some embodiments of the present disclosure first obtains the remote sensing image taken for the target building area and the building reference information corresponding to the above target building area. Here, the acquired remote sensing image is used as an image data source for building matching. The building reference information can be used as a basic data set for subsequent building matching, which is convenient for subsequent accurate matching. Then, the above remote sensing image is image preprocessed to generate a preprocessed remote sensing image to remove image noise in the remote sensing image and ensure the accuracy of subsequent building matching. Then, using the pre-trained building matching model, the building segmentation for the above preprocessed remote sensing image can be accurately and quickly performed to obtain a building segmentation image. Then, the object feature information set corresponding to each building segmentation object included in the above building segmentation image can be accurately extracted to facilitate the subsequent building matching. Furthermore, based on the above object feature information set and the above building reference information, the building matching for the above remote sensing image can be accurately performed to generate a building matching result. Finally, the above building matching result is visualized to display the matching result. In summary, based on the building matching model, the building segmentation image can be accurately and efficiently generated. On this basis, by accurately generating the object feature information set corresponding to the building segmentation image, accurate matching with the building reference information can be achieved, and efficient matching of the building can be achieved.

[0112] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a building matching device. These device embodiments are Figure 1 Corresponding to the method embodiments shown, the building matching device can be specifically applied to various electronic devices.

[0113] like Figure 2As shown, a building matching device 200 includes: an acquisition unit 201, a preprocessing unit 202, a segmentation unit 203, an extraction unit 204, an execution unit 205 and a visualization unit 206. The acquisition unit 201 is configured to acquire a remote sensing image taken for a target building area and building reference information corresponding to the target building area; the preprocessing unit 202 is configured to perform image preprocessing on the remote sensing image to generate a preprocessed remote sensing image; the segmentation unit 203 is configured to perform building segmentation on the preprocessed remote sensing image using a pre-trained building matching model to obtain a building segmentation image; the extraction unit 204 is configured to extract an object feature information set corresponding to each building segmentation object included in the building segmentation image; the execution unit 205 is configured to perform building matching on the remote sensing image according to the object feature information set and the building reference information to generate a building matching result; and the visualization unit 206 is configured to perform visualization processing on the building matching result.

[0114] It is understandable that the units described in the building matching device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the building matching device 200 and the units included therein, and will not be described in detail here.

[0115] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0116] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0117] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0118] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0119] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0120] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0121] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a remote sensing image taken for a target building area and building reference information corresponding to the target building area; performs image preprocessing on the remote sensing image to generate a preprocessed remote sensing image; uses a pre-trained building matching model to perform building segmentation on the preprocessed remote sensing image to obtain a building segmentation image; extracts an object feature information set corresponding to each building segmentation object included in the building segmentation image; performs building matching on the remote sensing image based on the object feature information set and the building reference information to generate a building matching result; and visualizes the building matching result.

[0122] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0123] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The described units may also be set in a processor, for example, it may be described as: a processor includes an acquisition unit, a preprocessing unit, a segmentation unit, an extraction unit, an execution unit and a visualization unit. The names of these units do not constitute a limitation on the units themselves in some cases. For example, the acquisition unit may also be described as "a unit for acquiring remote sensing images taken for a target building area and building reference information corresponding to the target building area".

[0125] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0126] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A building matching method based on remote sensing images, comprising: Acquire a remote sensing image taken for a target building area and building reference information corresponding to the target building area; Performing image preprocessing on the remote sensing image to generate a preprocessed remote sensing image; Using a pre-trained building matching model, performing building segmentation on the pre-processed remote sensing image to obtain a building segmentation image; Extracting an object feature information set corresponding to each building segmentation object included in the building segmentation image; Performing building matching for the remote sensing image according to the object feature information set and the building reference information to generate a building matching result; The building matching result is visualized.

2. The method according to claim 1, wherein: After acquiring the remote sensing image taken for the target building area and the building reference information corresponding to the target building area, the method further includes: Obtaining regional road distribution information corresponding to the target building area; Extracting road distribution feature information corresponding to the remote sensing image; According to the regional road distribution information and the road distribution feature information, the remote sensing image is cropped to obtain a cropped image; The cropped image is determined as a remote sensing image.

3. The method according to claim 1, wherein: The performing image preprocessing on the remote sensing image to generate a preprocessed remote sensing image includes: Performing radiation correction on the remote sensing image to generate a radiation corrected image; Performing atmospheric correction on the radiation-corrected image to generate an atmospheric-corrected image; Performing geometric correction on the atmospheric correction image to generate a geometric correction image; The preprocessed remote sensing image is generated according to the geometrically corrected image.

4. The method according to claim 1, wherein: The method of using the pre-trained building matching model to perform building segmentation on the pre-processed remote sensing image to obtain a building segmentation image includes: Inputting the preprocessed remote sensing image into a plurality of first convolutional layers connected in series included in the building matching model to generate first convolution information; Performing maximum pooling processing of a predetermined specification on the first convolution information to generate first pooling information; Inputting the first pooling information into a plurality of second convolutional layers connected in series included in the building matching model to generate second convolutional information; Performing a maximum pooling process of a predetermined specification on the second convolution information to generate second pooling information; Inputting the second pooling information into a plurality of third convolutional layers connected in series included in the building matching model to generate third convolutional information; Performing a maximum pooling process of a predetermined specification on the third convolution information to generate third pooling information; Inputting the third pooling information into a plurality of fourth convolutional layers connected in series included in the building matching model to generate fourth convolutional information; Performing a maximum pooling process of a predetermined specification on the fourth convolution information to generate fourth pooling information; Inputting the fourth pooling information into a plurality of fifth convolutional layers connected in series included in the building matching model to generate fifth convolutional information; Performing up-sampling processing on the fifth convolution information to generate first up-sampling information; Performing an average pooling process of a predetermined specification on the third convolution information to generate fifth pooling information; Inputting the fifth pooling information into a plurality of sixth convolutional layers connected in series included in the building matching model to generate sixth convolutional information; Generate first splicing information according to the sixth convolution information, the first up-sampling information and the fourth convolution information; Performing up-sampling processing on the first splicing information to generate second up-sampling information; Performing an average pooling process of a predetermined specification on the second convolution information to generate sixth pooling information; Inputting the sixth pooling information into a plurality of seventh convolutional layers connected in series included in the building matching model to generate seventh convolutional information; Generate second splicing information according to the seventh convolution information, the second up-sampling information and the third convolution information; performing up-sampling processing on the second splicing information to generate third up-sampling information; Performing an average pooling process of a predetermined specification on the first convolution information to generate seventh pooling information; Inputting the seventh pooling information into a plurality of eighth convolutional layers connected in series included in the building matching model to generate eighth convolutional information; Generate third splicing information according to the eighth convolution information, the third up-sampling information and the second convolution information; Performing up-sampling processing on the third splicing information to generate fourth up-sampling information; Generate fourth splicing information according to the fourth up-sampling information and the first convolution information; The fourth splicing information is input into a plurality of serially connected ninth convolutional layers included in the building matching model to generate a building segmentation image.

5. The method according to claim 4, wherein: The step of generating first splicing information according to the sixth convolution information, the first up-sampling information and the fourth convolution information includes: Inputting the fourth convolution information into a convolutional neural network based on an attention mechanism to generate first attention information; Inputting the sixth convolution information into the convolutional neural network based on the attention mechanism to generate second attention information; The first attention information, the first upsampling information, and the second attention information are concatenated to generate the first concatenated information.

6. The method according to claim 2, wherein: The step of cropping the remote sensing image according to the regional road distribution information and the road distribution feature information to obtain a cropped image includes: Using a regional road key feature point generation model, generating a first road key feature point information set corresponding to the regional road distribution information; Using the regional road key feature point generation model, a second road key feature point information set corresponding to the road distribution feature information is generated; Performing feature point information comparison on the first road key feature point information set and the second road key feature point information set to generate a comparison result; The remote sensing image is cropped according to the comparison result to obtain a cropped image.

7. The method according to claim 1, wherein: The method of using the pre-trained building matching model to perform building segmentation on the pre-processed remote sensing image to obtain a building segmentation image includes: Inputting the preprocessed remote sensing image into a plurality of serially connected tenth convolutional layers included in the building matching model to generate tenth convolutional information; Performing a maximum pooling process of a predetermined specification on the tenth convolution information to generate eighth pooling information; Inputting the eighth pooling information into a plurality of serially connected eleventh convolutional layers included in the building matching model to generate eleventh convolutional information; Performing a maximum pooling process of a predetermined specification on the eleventh convolution information to generate ninth pooling information; Performing pooling information fusion on the ninth pooling information and the eighth pooling information to generate first fusion information; Inputting the first fusion information into a plurality of twelfth convolutional layers connected in series included in the building matching model to generate twelfth convolutional information; Performing a maximum pooling process of a predetermined specification on the twelfth convolution information to generate tenth pooling information; Performing pooling information fusion on the tenth pooling information and the ninth pooling information to generate second fusion information; Inputting the second fusion information into a plurality of serially connected thirteenth convolutional layers included in the building matching model to generate thirteenth convolutional information; Performing a maximum pooling process of a predetermined specification on the thirteenth convolution information to generate an eleventh pooling information; Performing pooling information fusion on the eleventh pooling information and the tenth pooling information to generate third fusion information; Inputting the third fusion information into a plurality of fourteenth convolutional layers connected in series included in the building matching model to generate fourteenth convolutional information; Performing up-sampling processing on the fifteenth convolution information to generate fifth up-sampling information; Performing an average pooling process of a predetermined specification on the second fusion information to generate twelfth pooling information; Inputting the twelfth pooling information into a plurality of fifteenth convolutional layers connected in series included in the building matching model to generate fifteenth convolutional information; splicing the fifth up-sampling information, the fifteenth convolution information and the thirteenth convolution information to generate fifth splicing information; Performing up-sampling processing on the fifth splicing information to generate sixth up-sampling information; Performing an average pooling process of a predetermined specification on the first fusion information to generate thirteenth pooling information; Inputting the thirteenth pooling information into a plurality of series-connected sixteenth convolutional layers included in the building matching model to generate sixteenth convolutional information; splicing the sixth up-sampling information, the sixteenth convolution information and the twelfth convolution information to generate sixth splicing information; Performing up-sampling processing on the sixth splicing information to generate seventh up-sampling information; Performing an average pooling process of a predetermined specification on the tenth convolution information to generate fourteenth pooling information; Inputting the fourteenth pooling information into a plurality of seventeenth convolutional layers connected in series included in the building matching model to generate seventeenth convolutional information; splicing the seventh up-sampling information, the seventeenth convolution information and the eleventh convolution information to generate seventh splicing information; Performing up-sampling processing on the seventh splicing information to generate eighth up-sampling information; Splicing the eighth up-sampling information and the tenth convolution information to generate eighth splicing information; The eighth splicing information is input into a plurality of series-connected eighteenth convolutional layers included in the building matching model to generate a building segmentation image.

8. A building matching device based on remote sensing images, comprising: An acquisition unit is configured to acquire a remote sensing image taken for a target building area and building reference information corresponding to the target building area; a preprocessing unit configured to perform image preprocessing on the remote sensing image to generate a preprocessed remote sensing image; A segmentation unit is configured to perform building segmentation on the preprocessed remote sensing image using a pre-trained building matching model to obtain a building segmentation image; An extraction unit is configured to extract an object feature information set corresponding to each building segmentation object included in the building segmentation image; an execution unit, configured to execute building matching for the remote sensing image according to the object feature information set and the building reference information to generate a building matching result; The visualization unit is configured to perform visualization processing on the building matching result.

9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images