Township building identification method, classification method, device, electronic equipment and medium

Abstract

The present disclosure provides a village and town building identification method, a classification method, a device, an electronic device and a medium. The village and town building identification method comprises: constructing a region three-dimensional image of a village and town building region based on sample oblique photography data, the region three-dimensional image comprising a plurality of single village and town building three-dimensional sub-images; extracting facade image data of the plurality of single village and town building three-dimensional sub-images from the region three-dimensional image respectively; labeling a plurality of labels for each facade image data of the plurality of facade image data to obtain identification data of the plurality of single village and town building three-dimensional sub-images, the plurality of labels respectively representing a plurality of types of building features of the single village and town building three-dimensional sub-images; generating a target identification model according to the identification data of the plurality of single village and town building three-dimensional sub-images; and identifying building features of a plurality of single village and town buildings in a target region by using the target identification model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image recognition, in particular to a village building identification method, a village building classification method, a device, an electronic device and a storage medium. BACKGROUND

[0002] In land planning and design, it is often necessary to investigate single buildings in a region in order to evaluate and analyze the current land landscape, manage construction, and improve future land planning and building design.

[0003] At present, such researches are mostly focused on urban areas. The construction amount of village buildings is comparable to that of urban buildings, and there is almost no street view data of village buildings in existing data. Moreover, due to the diversity of the appearance of village buildings, it is inconvenient to obtain relevant information of village buildings and construct models. Therefore, the identification and feature analysis of village buildings have important application value in actual work. SUMMARY

[0004] In view of the above problems, the present disclosure provides a village building identification method, a village building classification method, a device, an electronic device and a storage medium.

[0005] According to a first aspect of the present disclosure, a village building identification method is provided, comprising: constructing a regional three-dimensional image of a village building region based on sample oblique photography data, the regional three-dimensional image comprising a plurality of single village building three-dimensional sub-images; extracting facade image data of the plurality of single village building three-dimensional sub-images from the regional three-dimensional image respectively; labeling a plurality of labels for each facade image data of the plurality of facade image data to obtain identification data of the plurality of single village building three-dimensional sub-images, the plurality of labels respectively representing a plurality of types of building features of the single village building three-dimensional sub-images; generating a target identification model according to the identification data of the plurality of single village building three-dimensional sub-images; and identifying building features of a plurality of single village buildings in a target region by using the target identification model.

[0006] According to an embodiment of the present disclosure, facade image data of a plurality of single-village-and-town building three-dimensional sub-images is extracted from a region three-dimensional image, including: obtaining three-dimensional point cloud data and an overhead image of the region three-dimensional image; projecting the overhead image onto the three-dimensional point cloud data to filter first point cloud data and second point cloud data of the plurality of single-village-and-town building three-dimensional sub-images from the three-dimensional point cloud data, respectively, the first point cloud data representing boundaries of the single-village-and-town building three-dimensional sub-images, and the second point cloud data representing textures of the single-village-and-town building three-dimensional sub-images; according to the first point cloud data, spatial structures of the plurality of single-village-and-town building three-dimensional sub-images are segmented from the region three-dimensional image; according to the second point cloud data, the spatial structures of the plurality of single-village-and-town building three-dimensional sub-images are respectively subjected to texture mapping to obtain the plurality of single-village-and-town building three-dimensional sub-images; and the facade image data of the plurality of single-village-and-town building three-dimensional sub-images is extracted, respectively.

[0007] According to an embodiment of the present disclosure, a plurality of labels are labeled for each facade image data of a plurality of facade image data to obtain identification data of a plurality of single-village-and-town building three-dimensional sub-images, including: a plurality of labels are labeled for each facade image data of the facade image data by using a plurality of labeling models, the plurality of labeling models being respectively used for labeling a plurality of types of labels; cross-validation is performed on the labeled facade image data by using the plurality of labeling models; and in a case where it is determined that the labeled facade image data passes the validation, a plurality of discrete variables are set for each facade image data of the plurality of facade image data according to the plurality of labels to obtain the identification data of the plurality of single-village-and-town building three-dimensional sub-images.

[0008] According to an embodiment of the present disclosure, the identification data of the plurality of single-village-and-town building three-dimensional sub-images includes training set data, validation set data, and test set data; and a target recognition model is generated according to the identification data of the plurality of single-village-and-town building three-dimensional sub-images, including: for each candidate recognition model in a plurality of candidate recognition models, the following operations are performed to obtain a plurality of test results: the candidate recognition model is trained by using the training set data to obtain a trained candidate recognition model; the trained candidate recognition model is validated by using the validation set data to obtain a validation result; and in a case where it is determined that the validation result is passed, the trained candidate recognition model is tested by using the test set data to obtain a test result for the candidate recognition model.

[0009] According to an embodiment of the present disclosure, the target recognition model is generated according to the identification data of the plurality of single-village-and-town building three-dimensional sub-images, and further includes: the target recognition model is determined from the plurality of candidate recognition models according to an optimal test result in the plurality of test results.

[0010] According to an embodiment of the present disclosure, the generating the target recognition model according to the identification data of the plurality of three-dimensional sub-images of the single village and town buildings further comprises: extracting door data, window data and facade image data of the plurality of single village and town buildings from the plurality of three-dimensional sub-images respectively by a target detector; and iteratively training the target recognition model by the door data, the window data and the facade image data in combination with a plurality of labels to obtain an optimized target recognition model.

[0011] According to an embodiment of the present disclosure, the target detector comprises a convolution layer, a target detection layer and a screening layer, and the extracting the door data, the window data and the facade image data of the plurality of single village and town buildings from the plurality of three-dimensional sub-images by the target detector comprises: extracting a plurality of initial feature data from the plurality of three-dimensional sub-images by the convolution layer; performing feature detection on the plurality of initial feature data by the target detection layer to obtain initial door data, initial window data and initial facade image data; and screening the door data, the window data and the facade image data from the initial door data, the initial window data and the initial facade image data respectively by the screening layer.

[0012] According to an embodiment of the present disclosure, the recognizing the single village and town buildings in the target region by the target recognition model comprises: obtaining oblique photography data of the target region; constructing a region three-dimensional image of the target region based on the oblique photography data of the target region; extracting facade image data of a plurality of single village and town buildings from the region three-dimensional image; and inputting the facade data into the target recognition model to obtain building characteristics of the plurality of single village and town buildings in the target region.

[0013] According to an embodiment of the present disclosure, the method further comprises: obtaining facade image data of the single village and town buildings and a plurality of labels associated with the single village and town buildings; and generating a visualization image according to the facade image data and the plurality of labels, the visualization image representing the building characteristics of the single village and town buildings.

[0014] A second aspect of the present disclosure provides a method for automatically classifying the properties of village and town buildings based on three-dimensional remote sensing images, comprising: obtaining image data of buildings in a study region; classifying and labeling the image data according to the properties of the village and town buildings to obtain a labeled database; constructing an image recognition model according to an image recognition algorithm and training the image recognition model by the labeled database; and identifying building images in a region to be classified by the trained image recognition model to obtain a building classification result of the region to be classified.

[0015] According to an embodiment of the present disclosure, the obtaining the image data of the buildings in the study region comprises: collecting video data of the study region by unmanned aerial vehicle oblique photography, performing super-resolution reconstruction, three-dimensional model generation and building facade extraction on the video data to obtain the image data of the buildings in the study region.

[0016] According to an embodiment of the present disclosure, the village and town building properties include: building functions, building layers, building quality, structural system, building style, building age, and whether abandoned.

[0017] According to an embodiment of the present disclosure, the method further comprises: adjusting parameters of the image recognition model according to the building classification result; or testing the image recognition model through test data, and adjusting parameters of the image recognition network according to the test result.

[0018] A third aspect of the present disclosure provides a village and town building identification device, comprising: a construction module configured to construct a region three-dimensional image of a village and town building region based on sample oblique photography data, the region three-dimensional image comprising a plurality of single village and town building three-dimensional sub-images; an extraction module configured to extract facade image data of the plurality of single village and town building three-dimensional sub-images from the region three-dimensional image respectively; a labeling module configured to label a plurality of labels to each facade image data of the plurality of facade image data to obtain identification data of the plurality of single village and town building three-dimensional sub-images, the plurality of labels respectively representing a plurality of types of building features of the single village and town building three-dimensional sub-images; a generation module configured to generate a target identification model according to the identification data of the plurality of single village and town building three-dimensional sub-images; and an identification module configured to identify building features of a plurality of single village and town buildings in a target region by using the target identification model.

[0019] A fourth aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the above-mentioned village and town building identification method or the automatic classification method of village and town building properties based on three-dimensional remote sensing images.

[0020] A fifth aspect of the present disclosure further provides a computer-readable storage medium having stored executable instructions, which are executed by a processor to cause the processor to perform the above-mentioned village and town building identification method or the automatic classification method of village and town building properties based on three-dimensional remote sensing images.

[0021] The present disclosure provides a village and town building identification method, which is based on image data carrying a plurality of labels, trains and optimizes a plurality of identification models, and determines a target identification model most suitable for village and town buildings. The target identification model obtained in this way not only focuses on a single attribute of a single building, but can also identify a plurality of attributes of a single building based on a plurality of labels. In addition, the door and window features of the village and town buildings are extracted by using a target detector, which can assist the target identification model in identifying the single village and town buildings and improve the identification accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0023] Figure 1A A flowchart of a method for identifying rural buildings according to embodiments of the present disclosure is schematically shown;

[0024] Figure 1B A schematic diagram of a three-dimensional sub-image of a single rural building according to embodiments of the present disclosure is schematically shown;

[0025] Figure 1C A schematic diagram of facade image data according to embodiments of the present disclosure is schematically shown;

[0026] Figure 2 A flowchart of extracting facade image data according to embodiments of the present disclosure is schematically shown;

[0027] Figure 3 A flowchart of labeling tags for facade image data according to embodiments of the present disclosure is schematically shown;

[0028] Figure 4A A flowchart of a method for automatically classifying the properties of rural buildings based on three-dimensional remote sensing images according to embodiments of the present disclosure is schematically shown;

[0029] Figure 4B A schematic diagram of a method for automatically classifying the properties of rural buildings based on three-dimensional remote sensing images according to embodiments of the present disclosure is schematically shown;

[0030] Figure 4C A schematic diagram of an image recognition model structure according to embodiments of the present disclosure is schematically shown;

[0031] Figure 4D A schematic diagram of a building classification result according to embodiments of the present disclosure is schematically shown;

[0032] Figure 5A A schematic diagram of a method for identifying rural buildings according to embodiments of the present disclosure is schematically shown;

[0033] Figure 5B A schematic diagram of a result of identifying the number of floors of a building according to embodiments of the present disclosure is schematically shown;

[0034] Figure 6 A block diagram of an identification device for rural buildings according to embodiments of the present disclosure is schematically shown; and

[0035] Figure 7 A block diagram of an electronic device for a method for identifying rural buildings according to embodiments of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0036] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to one skilled in the art that the embodiments can be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques have been omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0037] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to be limiting of the present disclosure. As used herein, the term "including" and variations thereof, such as "includes" and "included", mean that the components, steps, operations, and / or features so- described are present, but do not exclude the presence or addition of one or more other components, steps, operations, and / or features.

[0038] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings that are consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0039] In the case where expressions such as "at least one of A, B, and C, etc." are used, it should be generally interpreted to include one or more of the items enumerated in the expression (e.g., "a system having at least one of A, B, and C" should include, but not be limited to, a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, etc.).

[0040] Embodiments of the present disclosure provide a method for identifying rural buildings, comprising: constructing a region three-dimensional image of a rural building region based on sample oblique photography data, the region three-dimensional image comprising a plurality of single rural building three-dimensional sub-images; extracting facade image data of the plurality of single rural building three-dimensional sub-images from the region three-dimensional image, respectively; labeling a plurality of labels for each of the plurality of facade image data to obtain identification data of the plurality of single rural building three-dimensional sub-images, the plurality of labels respectively representing a plurality of types of building features of the single rural building three-dimensional sub-images; generating a target identification model according to the identification data of the plurality of single rural building three-dimensional sub-images; and identifying building features of a plurality of single rural buildings in a target region by using the target identification model.

[0041] Figure 1A A flowchart of a method for identifying rural buildings according to an embodiment of the present disclosure is schematically shown. Figure 1B A schematic view of a single rural building three-dimensional sub-image according to an embodiment of the present disclosure is schematically shown. Figure 1CA schematic diagram of facade image data according to an embodiment of the present disclosure is shown.

[0042] As shown in Figure 1A The method for identifying rural buildings of this embodiment includes operations S110-S150.

[0043] In operation S110, a regional three-dimensional image of a rural building area is constructed based on sample oblique photography data.

[0044] The sample oblique photography data can come from unmanned aerial vehicle (UAV) oblique photography. Because the distribution of rural buildings is complex and disordered, the flexible, efficient, and low-labor-cost characteristics of UAVs can be used to quickly and comprehensively obtain the topographic features of rural buildings.

[0045] Oblique image data can reflect the actual situation of buildings from multiple observation angles, and the side texture of buildings can also be collected using oblique images. The amount of oblique photography data is small, making it easy to store and operate. After image processing, such as light analysis and geometric correction, and other operations, the oblique photography data can be used to generate a three-dimensional model, such as a three-dimensional image, by using special surveying and mapping software.

[0046] The regional three-dimensional image can include multiple monomer rural building three-dimensional sub-images. For example, a rural building area can be the building area of a village, and the corresponding regional three-dimensional image can be a three-dimensional image of the building area of the village. A monomer rural building can be each residential building in the village, and the corresponding monomer rural building three-dimensional sub-image can be a three-dimensional image of the residential building.

[0047] The UAV has the characteristics of high-precision positioning. For example, a high-precision real-time kinematic (RTK) module is mounted on the UAV. For example, a small number of ground positioning reference points are used to realize the positioning of the regional three-dimensional model in geographical space or the spatial positioning of monomer rural buildings.

[0048] Image reconstruction algorithms can be used to achieve super-resolution reconstruction of UAV images, enhance the details of the three-dimensional image, and make the three-dimensional image better applied to subsequent image extraction tasks. The reconstructed three-dimensional model can also meet the requirements of re-measurement, for example, compared with manual surveying, the use of a three-dimensional model can meet the needs of re-checking the three-dimensional model and performing high-precision spatial measurements such as size and distance.

[0049] In operation S120, facade image data of multiple monomer rural building three-dimensional sub-images is extracted from the regional three-dimensional image.

[0050] For example, the process of extracting facade image data can include first segmenting each monomer rural building three-dimensional sub-image from the regional three-dimensional image, for example,Figure 1B The three-dimensional sub-image of the single rural building is shown, and the facade image data is extracted from the three-dimensional sub-image, for example Figure 1C The facade image data of the single rural building is shown.

[0051] At least one facade image data can be extracted from the three-dimensional sub-image of each single rural building. For example, it can be selected to extract the facade image data of the front view perspective from the three-dimensional sub-image of each single rural building. It can also be selected to extract the facade image data of multiple perspectives from the three-dimensional sub-image of each single rural building, such as the facade image data of front view, rear view, top view, etc.

[0052] The facade image data can be building facade data, which can represent the interface directly contacted by the building and the external space of the building, for example, the building facade data can include the building roof part and the building wall peripheral part. The facade image data can represent the structure and external topography of the building.

[0053] In the embodiments of the present disclosure, the building facade is extracted from the three-dimensional sub-image of the single rural building, which can realize the collection of building images that are not easy to enter and investigate, for example, idle uninhabited courtyards, closed courtyards or structurally unsafe buildings, etc.

[0054] In operation S130, a plurality of labels are labeled on each facade image data of the plurality of facade image data, and identification data of the plurality of single rural building three-dimensional sub-images is obtained, and the plurality of labels respectively represent a plurality of types of building features of the single rural building three-dimensional sub-image.

[0055] Since the rural buildings are mainly self-built houses, compared with urban buildings, the building differences of rural buildings are larger. Therefore, it is necessary to label the building features of single rural buildings from multiple angles. In addition, through multiple labels, the building features of single rural buildings can be more comprehensively described.

[0056] For example, the identification data of the single rural building three-dimensional sub-image can be facade image data carrying multiple labels, or function data converted from the facade image data and the labels, or text data describing the facade image data and the labels.

[0057] The setting of multiple labels can meet the multiple needs of building surveying in actual surveying of rural buildings, so as to describe the features of rural buildings from multiple angles. In addition, on the basis of meeting the basic description needs, the labels are expanded according to actual needs to realize the effect of adapting to local conditions.

[0058] In operation S140, a target recognition model is generated according to the identification data of the plurality of single rural building three-dimensional sub-images.

[0059] For example, the identification data of the plurality of monomer village and town building three-dimensional sub-images is taken as a database, and a neural network is designed and trained, so that a target identification model for identifying monomer village and town buildings can be obtained.

[0060] For another example, the identification data of the plurality of monomer village and town building three-dimensional sub-images is taken as a database, and an existing neural network is trained and optimized, so that a target identification model for identifying monomer village and town buildings can also be obtained.

[0061] The neural network has the characteristic of continuous learning, and the target identification model can be iteratively optimized multiple times as the amount of training sample data increases. The accuracy of the target identification model can be further improved as the amount of data increases.

[0062] In operation S150, the building features of the plurality of monomer village and town buildings in the target region are identified by using the target identification model.

[0063] In the embodiments of the present disclosure, the target identification model is trained by using the identification data of the monomer village and town building three-dimensional sub-images, and the monomer village and town buildings in the village and town building region can be identified based on the building features of the monomer village and town buildings, so as to obtain the relevant information of the monomer village and town buildings, thereby facilitating data analysis of the individual village and town buildings.

[0064] According to the embodiments of the present disclosure, the identification model suitable for monomer village and town building identification is trained based on the identification data of the monomer village and town buildings carrying multiple labels. The target identification model obtained in this way not only focuses on the single attribute of the monomer building, but also can identify the multiple attributes of the monomer building based on the multiple labels. In addition, the use of unmanned aerial vehicle oblique photography technology to collect sample oblique photography data can shorten the collection of sample data, improve the efficiency of complete data collection, and enhance the applicability.

[0065] Figure 2 A flowchart for extracting facade image data according to an embodiment of the present disclosure is schematically shown.

[0066] As shown in Figure 2 Operation S120 includes operations S210 to S250.

[0067] In operation S210, the three-dimensional point cloud data and the overhead image of the region three-dimensional image are obtained.

[0068] In operation S220, the overhead image is projected onto the three-dimensional point cloud data, so as to respectively filter out the first point cloud data and the second point cloud data of the plurality of monomer village and town building three-dimensional sub-images from the three-dimensional point cloud data.

[0069] At operation S230, according to the first point cloud data, a spatial structure of each monomer village and town building three-dimensional sub-image is segmented from the area three-dimensional image.

[0070] At operation S240, according to the second point cloud data, the spatial structure of each monomer village and town building three-dimensional sub-image is respectively textured to obtain a plurality of monomer village and town building three-dimensional sub-images.

[0071] At operation S250, facade image data of the plurality of monomer village and town building three-dimensional sub-images is respectively extracted.

[0072] In the embodiments of the present disclosure, the three-dimensional point cloud data can be obtained by photographing when the unmanned aerial vehicle flies around the village and town building at 360°. The overhead image is an overhead view taken by the unmanned aerial vehicle at an overhead angle. For example, the three-dimensional point cloud data can be the overall three-dimensional point cloud data of the entire village and town building area, or can be a collection of three-dimensional point cloud sub-data of a plurality of monomer village and town buildings in the village and town building area. The three-dimensional point cloud data can include point cloud data describing the building structure and point cloud data describing the building appearance.

[0073] The overhead image is randomly projected onto the three-dimensional point cloud data, and in combination with the assistance of multiple projection views, the first point cloud data and the second point cloud data of each monomer village and town building three-dimensional sub-image can be screened from the three-dimensional point cloud data, thereby realizing the segmentation of the monomer village and town building three-dimensional sub-image from the area three-dimensional image of the village and town building. The first point cloud data can represent the boundary of the monomer village and town building three-dimensional sub-image, and is used to describe the building structure. The second point cloud data can represent the texture of the monomer village and town building three-dimensional sub-image, and is used to describe the building appearance.

[0074] For example, according to the first point cloud data, the boundary of the monomer village and town building can be described, thereby constructing the spatial structure of the monomer village and town building three-dimensional sub-image. The spatial structure can be a three-dimensional white model of the monomer village and town building. The white model can describe the structural characteristics of the monomer village and town building, such as the number of building floors, the building area, and the building shape.

[0075] For example, according to the second point cloud data, the texture of the spatial structure of the monomer village and town building can be described. The texture can describe the appearance characteristics of the monomer village and town building, such as the building style, the building system, and the building age. The texture mapping for the white model can form the appearance of the monomer village and town building, thereby constructing the monomer village and town building three-dimensional sub-image.

[0076] Through the embodiments of the present disclosure, the three-dimensional sub-image of the single village and town building is first segmented from the three-dimensional image of the region, and then the corresponding facade image data of the single village and town building is extracted from the three-dimensional sub-image, so that the partial missing of the facade image data of the single village and town building can be avoided, and the problem of difficult segmentation between the facade image data of the two single village and town buildings adjacent thereto can be avoided. In addition, through the method combining the three-dimensional point cloud data and the overhead image, the point cloud data corresponding to each single village and town building in the three-dimensional point cloud data can be quickly and accurately located, and the extraction rate and extraction effect of the facade image data are improved. Through the three-dimensional point cloud segmentation method of multiple projection views, the correlation between the two-dimensional views such as the overhead image and the two-dimensional views is used, and the need for correction of adjacent point clouds is avoided, so that the three-dimensional point cloud entity can be segmented faster, and the speed of single building segmentation is further improved.

[0077] Figure 3 A flowchart of labeling a facade image data tag according to an embodiment of the present disclosure is schematically shown.

[0078] As Figure 3 shown, the step of labeling multiple tags for each facade image data of the multiple facade image data to obtain the identification data of the multiple single village and town building three-dimensional sub-images in operation S130 includes operations S310-S350.

[0079] In operation S310, multiple labeling models are used to label multiple tags for each facade data of the facade image data.

[0080] In operation S320, the multiple labeling models are used to cross-verify the labeled facade image data.

[0081] In operation S330, in a case where it is determined that the labeled facade image data passes the verification, multiple discrete variables are set for each facade image data of the multiple facade image data according to the multiple tags, and the identification data of the multiple single village and town building three-dimensional sub-images is obtained.

[0082] In the embodiments of the present disclosure, the labeling model can be a neural network after deep learning. The multiple labeling models are respectively used to label multiple types of tags.

[0083] Each labeling model can be relatively good at labeling a certain specific type of tag. For example, each labeling model can be relatively deep in deep learning for labeling a certain specific type of tag, and extensive deep learning is performed for labeling other types of tags. Each labeling model can also be good at labeling multiple types of tags. For example, each labeling model can be deep learning for labeling multiple types of tags.

[0084] It should be noted that each annotation model also needs to be annotated for multiple types of label annotation to ensure that each annotation model can also annotate non-expert types of labels and ensure applicable accuracy.

[0085] For example, the annotation speed of the annotation model for the expert type of label is higher than that for the non-expert type of label, and the annotation accuracy for the expert type of label is similar to that for the non-expert type of label.

[0086] For example, the plurality of annotation models includes three annotation models. The plurality of extracted facade image data can be sequentially input into the three annotation models, and the three annotation models can only annotate the labels of their own expert types for each facade image data.

[0087] During the label annotation process, the three annotation models perform non-repeated type label annotation on each facade image data. For example, it is assumed that the first annotation model and the second annotation model are both good at annotating type A labels. In the case where it is determined that the first annotation model has annotated the type A label for the facade image data, when the annotated facade image data is input to the second annotation model, the second annotation model no longer annotates the type A label.

[0088] During the cross-validation process, the three annotation models cross-validate the labels of the plurality of facade image data, and each annotation model can verify whether the non-annotated label passes. The verification process can include sequentially inputting the annotated facade image data into the three annotation models, and each annotation model re-annotates the non-annotated label type.

[0089] In the case where it is determined that the re-annotated label is consistent with the previously annotated label, it is considered that the label annotation of this type is verified and the previously annotated label is retained. In the case where it is determined that the re-annotated label is inconsistent with the previously annotated label, it is considered that the label annotation of this type is not verified, and the facade image data is recorded and waits for the next round of re-annotation by other annotation models. After completing the re-annotation process, cross-validation is again performed.

[0090] In the case where the verification is passed, a plurality of discrete variables also need to be set for each facade image data of the plurality of facade image data according to the annotated label. For example, the annotated label can be text data, and therefore a discrete variable corresponding to the label needs to be set for each facade data, so that the label is converted from text data to machine language that can be recognized by a neural network model.

[0091] The disclosure provides an embodiment of a multi-type label to schematically illustrate a plurality of types of labels. The multi-type label can be used to quantitatively describe the architectural properties of a building. For example, the quantitative types can include descriptive, historical, perceptual, and rational cognitive. For example, the plurality of types of labels can include building function, building story, building quality, structural system, building style, building age, and whether abandoned.

[0092] It should be noted that the seven types of labels provided by the disclosure are only illustrative. The disclosure can quantitatively describe the architectural characteristics of single-village and town buildings from seven angles. Those skilled in the art can extend the types of labels according to actual description needs. The disclosure does not limit the number and types of labels.

[0093] Exemplarily, the interpretation of the plurality of types of labels is shown in Table 1.

[0094] Table 1

[0095]

[0096] For example, the label annotated by the annotation model for the building function of the elevation image data is a residential building, so the discrete variable of this type of label for the elevation image data is set to 0. The label annotated by the annotation model for the building style of the elevation image data is that the coordination with the surrounding style is very poor, so the discrete variable of this type of label for the elevation image data is set to 2.

[0097] Compared with urban buildings, due to the large differences in structure and appearance of village and town buildings, a single type of label cannot comprehensively describe the characteristics of village and town buildings. For example, the basic characteristics of village and town buildings can be expressed by building function and building story labels. The residential environment characteristics of village and town buildings can be described by building abandonment discrimination, building age, and building quality labels, so as to determine whether the village and town building needs to be repaired or demolished according to the building abandonment discrimination, building age, and building quality labels. The planning characteristics and architectural style of village and town buildings can be described by building style labels, so as to manage village and town planning according to the building style labels.

[0098] For example, two adjacent single-village and town buildings in a region can both be three-story residential buildings, but the first single-village and town building is in an abandoned state, and the second single-village and town building is in a used state. If only based on building story and building function for annotation, the characteristics of the first single-village and town building in the "abandoned state" can be ignored, resulting in missing the best repair period for the building.

[0099] In the embodiments of the present disclosure, the label can further include a door position and a window position. For example, the door position is described by door coordinates, and the window position is described by window coordinates. The door position and the window position can be used to assist in identifying the characteristics of the single-family village building. For example, in a case where a label of two window positions is labeled for the facade image data, and it is determined that the two windows are arranged along the vertical ground direction through the coordinates of the two window positions, it can be considered that the building of the single-family building has two floors.

[0100] According to the embodiments of the present disclosure, the characteristics of the single-family building are described from multiple aspects by using multiple types of labels, which can not only improve the identification accuracy of the single-family buildings in the same area, but also achieve fine-grained feature recognition, fully utilize the difference of the characteristics of the single-family buildings, and thus increase the use value of the identification result.

[0101] Figure 4A A flowchart of a method for automatically classifying village building properties based on three-dimensional remote sensing images according to an embodiment of the present disclosure is schematically shown. Figure 4B A schematic diagram of a method for automatically classifying village building properties based on three-dimensional remote sensing images according to an embodiment of the present disclosure is schematically shown.

[0102] As shown in Figure 4A The method for automatically classifying village building properties based on three-dimensional remote sensing images includes operation S410 to operation S440.

[0103] In operation S410, image data of buildings in a study area is obtained.

[0104] The embodiments of the present disclosure can provide a convenient, time-saving, and easy-to-operate workflow for rural building monitoring. In the design of the workflow, the regional differences of building environment in rural and urban areas are also considered. First, image data of buildings in a study area is obtained. The study area can be a rural area.

[0105] Optionally, obtaining the image data of the buildings in the study area includes: collecting video data of the study area by unmanned aerial vehicle oblique photography, performing super-resolution reconstruction, three-dimensional model generation, and building facade extraction on the video data, and obtaining the image data of the buildings in the study area. For example, unmanned aerial vehicle oblique photography can be used to obtain original data of the study area. The original data can be video data, and corresponding image data can be collected from the video data. The unmanned aerial vehicle has the advantages of flexibility, efficiency, and low labor cost. In the embodiments of the present disclosure, aerial survey images of four villages (Daju Geachzang, Xiaochuanfangyu, Xiguoyuan, and Taohuasi) in Jizhou District, Tianjin, China are obtained. The aerial survey area is 8.7 square kilometers, and the resolution accuracy is less than 10 cm. For the obtained aerial photography data, super-resolution reconstruction, three-dimensional model generation, and building facade extraction are performed on the aerial photography data.

[0106] In operation S420, the image data is classified and labeled according to the rural building properties, and a labeled database is obtained.

[0107] For example, the rural building properties include building function, building floor number, building quality, structural system, building style, building age, and whether abandoned. For the cognitive level of buildings, it can generally be divided into four levels of descriptive, historical, perceptual, and rational cognition. According to the classification of building feature elements in classic literature in the field of architecture and the needs of rural development, the present disclosure focuses on analyzing the following seven features of rural buildings: building function, building floor number, building quality, structural system, building style, building age, and whether abandoned.

[0108] According to the rural building features in the data preprocessing step, each group of building monomers is numbered, three groups of professionals are invited, and after unified training, the artificial discrete variable labeling of each index of building quality is performed according to Table 1. In order to reduce uncertainty, it is ensured that each expert only labels all pictures of one type of building feature, and the remaining experts cross-check. For pictures with different labels, an agreement is reached through discussion, and finally a labeled training library is obtained.

[0109] In operation S430, an image recognition model is constructed according to an image recognition algorithm, and the image recognition model is trained through the labeled database.

[0110] Model construction - comparative evaluation and selection of the best model: considering the sample size and rural building data characteristics, the present disclosure selects four network structures that have better performance in current image classification competitions: Resnet50, Resnet101, Xception, and EfficientNet_B7, as shown in Figure 4CThe models shown (from left to right: Xception, Resnet50, Resnet101, EfficientNet) are outstanding in processing large-scale image recognition classification, for example: Resnet solves the problem of increasing traditional CNN to solve the degradation caused by excessive network depth due to gradient disappearance, and solves the loss and loss of information by constructing an identity mapping between convolutional layer blocks, and solves the loss and loss of information in the case of expanding the network depth. In this experiment, Resnet50 and Resnet101 with different depths under the Resnet network were selected for testing. The network structure of Xception and Inception has the same focus: compared with the deepening of the network structure of Resnet, the neural network is expanded without increasing the calculation cost. Xception is more advanced than Inception: completely separate the cross-channel correlation and spatial correlation without joint mapping. And also achieved good results. The EfficentNet model proposes a new model scaling method, which is different from the traditional method of arbitrarily scaling network dimensions. It can obtain the optimal set of parameters based on neural network search technology. EfficentNet has an efficient compound coefficient to enlarge the network from three dimensions of network depth, width and input image resolution. Through transfer learning, EfficientNets has reached the most advanced level on multiple well-known datasets.

[0111] Due to the unevenness of the sample caused by too many buildings and too many residential buildings at the rural scale, the data collected is often unbalanced. Therefore, the CNN architecture selected in the present disclosure is optimized.

[0112] In the design of the loss function, the focal loss function is adopted. The focal loss solves the model performance problem caused by sample imbalance from the perspective of sample classification difficulty, and has a large penalty degree for simple samples, which can achieve the effect of concentrating on optimizing difficult and few examples. In data organization, the under-sampling method is adopted. By clustering, the most summarized cluster is selected and a part of the data in the cluster is randomly discarded. For few-sample data, the strategy of extraction and reinsertion is adopted in each batch to ensure the balance of data in each batch.

[0113] In the present disclosure, ACC, Recall and F1 are selected to evaluate the precision of CNN architecture automatic identification classification in both generalization and practical application scenarios. In order to ensure the stability of the reported model, the commonly used method is adopted, that is, the dataset is randomly divided into three parts: training (60%), validation (20%) and testing (20%). The splitting is carried out in a hierarchical manner so that the class ratio can be maintained in all splits, and each data distribution has different training, validation and test subsets randomly generated from the original dataset.

[0114] The performance of the four model architectures on the seven building feature classifications is shown in Table 2. According to the preferred results of the model, the most suitable model architecture is selected as Resnet 50.

[0115] Table 2

[0116]

[0117] Note: The bold in the table indicates the best performance among the four models.

[0118] In operation S440, the building image in the region to be classified is identified by the trained image recognition model, and the building classification result of the region to be classified is obtained.

[0119] After the above-mentioned model training is completed, a single administrative village is automatically identified and classified.

[0120] Based on the needs of practical rural building evaluation and monitoring and other application scenarios, a single administrative village (Dajujiazhuang) sample in the dataset is selected as the test object, and the other three village samples are used for training. Finally, the classification accuracy and feasibility of the preferred model (Resnet 50) are evaluated. This scenario is a simulation of the actual working scenario and is more practical in practice. The accuracy results of the instance test of Dajujiazhuang are shown in Table 3. Each index reaches more than 0.8, indicating that the prediction results of Resnet 50 for specific administrative villages are excellent. This paper only shows the most intuitive and most practical accuracy results. Figure 4D

[0121] The model classification results are automatically output into the actual required plan drawings by using ArcGIS software. By spatial positioning of the building vector contour and the model prediction result, 7 groups of building feature plan drawings are output, as shown in Table 4. Figure 4D

[0122] ​​Optionally, in the embodiments of the present disclosure, the method for automatically classifying rural building properties based on three-dimensional remote sensing images further comprises: adjusting the parameters of the image recognition model according to the building classification result; or testing the image recognition model through test data, and adjusting the parameters of the image recognition network according to the test result.

[0123] In order to make the model classification effect better, the image recognition model is continuously optimized.

[0124] The workflow proposed in the present disclosure can greatly shorten the time required for rural housing survey and is easy to operate. Taking 941 villages in Jizhou District of Tianjin as an example, the average number of buildings in each village is 200. In the traditional field survey method, according to experience, each surveyor can survey an average of 80 buildings per day. A traditional research team of ten people needs 235 days to complete the data sample collection work in the study area. A team of two people using unmanned aerial image collection can complete an average of 8 village data collection per day. A research team of ten people using the method in this paper can complete all information collection work in only 24 days. Only in the aspect of data collection, the time can be reduced by 10 times. The method proposed in the present disclosure can also greatly reduce the working time of data processing.

[0125] The method of the present disclosure can quickly obtain an overall evaluation of a village, or focus on the evaluation of a single attribute. Through this method, it can help the relevant part to have a macroscopic understanding of the overall situation of the villages in the region, and can be used as the basis for the follow-up in-depth research. Moreover, in some studies, a single attribute needs to be investigated. However, even if only the single building feature of whether the house is abandoned is studied, the traditional method also needs the survey personnel to evaluate each building. Therefore, the method proposed in the present disclosure can also greatly improve the efficiency of special investigation.

[0126] Through the standardized data processing process, the present disclosure can facilitate the promotion in different regions, and can also meet the data iteration and updating needs in the same region. In this paper, the model optimization is carried out by using the annotation training library, and the Resnet 50 model architecture most suitable for rural buildings in North China is selected. Since more actual needs in actual work are to correspond the research situation with the plan paper, finally the cooperation between the model output result and the ArcGIS geographic information platform to complete the plan paper output is shown. The complete working process from the collection of rural building feature collection to the final output is constructed, and the working process can be adjusted and standardized according to the research area. Using the working process can greatly improve the efficiency of rural building survey, and can assist the formulation and precise implementation of relevant policies.

[0127] According to the method for automatically classifying rural buildings based on three-dimensional remote sensing images, image data of buildings in a research area is obtained; the image data is classified and labeled according to the properties of the rural buildings, to obtain a labeled database; an image recognition model is constructed according to an image recognition algorithm, and the image recognition model is trained through the labeled database; and the trained image recognition model is used to recognize building images in a region to be classified, to obtain a building classification result of the region to be classified. The method selects seven rural building features with distinguishing degrees and descriptions, invites professionals to perform discrete artificial assignment, and then uses the results as samples for machine learning, to quickly and easily obtain overall building feature conditions of a village in the form of a single building attribute table and a seven-feature plan of the village, so that the classification efficiency is high and the accuracy is high, and the method can be used for monitoring and identifying changes in building features of a specified village.

[0128] Figure 5A A schematic diagram of a method for identifying rural buildings according to an embodiment of the present disclosure is shown.

[0129] As shown in Figure 5A , the method for identifying rural buildings includes data collection 510, data preprocessing 520, model selection 530, instance verification 540, and graphical output 550.

[0130] In the embodiments of the present disclosure, the data collection 510 is similar to the operation S210 to operation S250 of the present disclosure, and will not be described again. The data preprocessing 520 is similar to the operation S310 to operation S330, and will not be described again.

[0131] The model selection 530 can perform the step of generating the target recognition model according to the identification data of the plurality of single rural building three-dimensional sub-images in operation S140.

[0132] In the case where the identification data of the plurality of single rural building three-dimensional sub-images includes training set data, verification set data, and test set data, the step of generating the target recognition model according to the identification data of the plurality of single rural building three-dimensional sub-images in operation S140 includes: for each candidate recognition model in the plurality of candidate recognition models, performing the following operations to obtain a plurality of test results: training the candidate recognition model using the training set data to obtain a trained candidate recognition model; verifying the trained candidate recognition model using the verification set data to obtain a verification result; and in the case where it is determined that the verification result is passed, testing the trained candidate recognition model using the test set data to obtain a test result for the candidate recognition model. And determining the target recognition model from the plurality of candidate recognition models according to the optimal test result in the plurality of test results.

[0133] In the embodiments of the present disclosure, the candidate recognition model with the optimal recognition performance test result is selected as the target recognition model by training and optimizing a plurality of candidate recognition models. The plurality of candidate recognition models can include Resnet50, Resnet101, Xception and EfficientNet_B7 network structures.

[0134] The training data set is used for deep learning training of the candidate recognition model, so that the trained candidate recognition model can recognize the architectural features of each single village and town building. The verification data set is used for verification and optimization of the candidate recognition model. For example, in the case that the trained candidate recognition model passes the verification, the trained candidate recognition model can be tested by using the test data set to test the recognition performance of the trained candidate recognition model. In the case that the trained candidate recognition model does not pass the verification, the candidate model is optimized and trained again until the verification passes. Thus, it can be avoided that the trained candidate recognition model produces a large error in actual application. The test data set is used for testing the recognition performance of the trained candidate recognition model to test the accuracy of the trained candidate recognition model.

[0135] In the embodiments of the present disclosure, the step of generating the target recognition model according to the identification data of the plurality of single village and town building three-dimensional sub-images in operation S140 further includes: extracting door data, window data and facade image data of the plurality of single village and town buildings from the plurality of three-dimensional sub-images by a target detector; and iteratively training the target recognition model by combining the plurality of labels and the door data, the window data and the facade image data to obtain an optimized target recognition model.

[0136] The door data, the window data and the facade image data can assist in recognizing the architectural features of the single village and town building. For example, the door data and the window data can respectively include door coordinates and window coordinates. The door coordinates, the window coordinates and the facade image data can be combined to determine the arrangement position of the door and the window in the single village and town building, so as to recognize the building layers of the single village and town building. For example, the door data and the window data can respectively include door styles and window styles. The door styles, the window styles and the facade image data can be combined to determine the architectural style in the single village and town building, so as to recognize the architectural style of the single village and town building.

[0137] For example, the target detector can be implemented based on a Yolo network, for example, a Yolov5 network. The Yolo network detects target objects in an image by a bounding box, and directly outputs the position of the regression bounding box and the category to which the bounding box belongs. The Yolo network can achieve an operation speed of 45 frames per second, which can meet the real-time requirement.

[0138] The target detector implemented based on the Yolo network can include three parts: a convolutional layer, a target detection layer, and a screening layer. The screening layer can be a non-maximum suppression (NMS) screening layer.

[0139] In the embodiments of the present disclosure, the target detector is used to extract door data, window data, and facade image data of the single-village-and-town building from the plurality of three-dimensional sub-images, respectively, including:

[0140] The convolutional layer is used to extract a plurality of initial feature data from the plurality of three-dimensional sub-images. The target detection layer is used to perform feature detection on the plurality of initial feature data to obtain initial door data, initial window data, and initial facade image data. The screening layer is used to screen the door data, the window data, and the facade image data from the initial door data, the initial window data, and the initial facade image data, respectively.

[0141] For example, the convolutional layer can adopt a Google inceptionV1 network with 20 layers. The convolutional layer is used for feature extraction to improve the generalization ability of the model. The convolutional kernel can be a 1x1 convolutional kernel connected in parallel with a 3x3 convolutional kernel.

[0142] For example, the target detection layer can include 4 convolutional layers and 2 fully connected layers. The initial door data, the initial window data, and the initial facade image data can generate an output of 7x7x30 after being input into the target detection layer. The 4 convolutional layers can improve the generalization ability of the model.

[0143] For example, the NMS screening layer can screen the most suitable bounding box from a plurality of bounding boxes. For example, the bounding boxes with scores lower than a threshold value are filtered out first, and then the remaining bounding boxes are subjected to NMS non-maximum suppression to remove the bounding boxes with high overlap, thereby obtaining the most suitable bounding box and the category of the bounding box. The loss function can include position error, confidence error, and classification error.

[0144] In the case where the target detector cannot detect the door and window data, a target recognition model can be directly used to recognize the architectural features of the single-village-and-town building.

[0145] In the case that the door and window data can be detected by the target detector, the target detector and the target recognition model can be combined to identify the single village and town building, and the building characteristics of the single village and town building are obtained. For multiple single buildings, based on the identified building characteristics, the multiple single buildings can be classified, so that the village and town buildings of the same class have the same building characteristics. It should be noted that the same village and town building can be classified into multiple categories based on multiple building characteristics. In addition, the multilayer perceptron (MLP) can be used to perform convolution calculation again on the classification result to obtain a more accurate classification result.

[0146] The multilayer perceptron can be an artificial neural network (ANN). The multilayer perceptron can be used to input multiple groups of images to automatically identify and classify.

[0147] For example, the multilayer perceptron can be a two-layer fully connected multilayer perceptron, the number of hidden units is 1024, and the learning rate and other hyperparameters can be debugged. The weights of the input data of the multilayer perceptron can be initialized with random parameters and automatically learned through training. The sigmoid or tanh continuous function is used to simulate the response of neurons in the multilayer perceptron to excitation.

[0148] In the embodiments of the present disclosure, whether the labeled facade image data contains door and window information is detected by the yolo target detector. In the case that the facade image data includes door and window information, the image data of the door, the window, and the elevation view (facade image) is respectively convolved by the target recognition model to obtain the building characteristic identification result of the single village and town building. Based on the identification result, multiple single village and town buildings are classified, and the classification result can be x1, x2, and x3. The classification result (x1, x2, and x3) is used as the input data of the multilayer perceptron. The activation function can be a softmax function. The weights of the input data can be initialized with random parameters and automatically learned through training. After the operation of the multilayer perceptron, more accurate classification results of multiple single village and town buildings can be output.

[0149] The present disclosure also tests the identification performance of the combination of the target recognition model, the target detector, and the multilayer perceptron, and the test results are shown in Tables 3 and 4. The test data shown in Table 3 is the identification data generated from the sample oblique photography data, and the identification data includes training set data, validation set data, and test set data in a ratio of 8:1:1. The test data shown in Table 4 is newly sampled village oblique photography data. The identification objects include seven types of building characteristics, including building function, building layer, building quality, building structure, building style, building age, and whether abandoned.

[0150] Table 3

[0151]

[0152] Table 4

[0153]

[0154] As shown in Table 3, the target recognition module Resnet50 combined with the target detector yolov5 can achieve a test accuracy of 85% for the seven types of labels. Among them, the building layers, building structure, building style, building age and whether abandoned reach 90%, the building quality and building function reach 87%, and the comprehensive strength performs best. As shown in Table 4, the accuracy of building layers, building quality, building structure, building age and whether abandoned reaches 90%.

[0155] The recognition model combined with the target recognition module Resnet50 and the target detector yolov5 can complement each other with the single target recognition module Resnet50 to achieve more accurate classification prediction for the seven types of labels.

[0156] In the embodiments of the present disclosure, the instance verification 540 can perform the step of identifying the building characteristics of the single village and town building in the target area by using the target recognition model in operation S150, which includes:

[0157] Obtaining oblique photography data of the target area; constructing a regional three-dimensional image of the target area based on the oblique photography data of the target area; extracting facade image data of a plurality of single village and town buildings from the regional three-dimensional image; and inputting the facade data into the target recognition model to obtain the building characteristics of the plurality of single village and town buildings in the target area.

[0158] For example, the oblique photography data of the target area can be the oblique photography data of the to-be-detected area, which can be obtained by oblique photography of a drone.

[0159] The target area can also include a plurality of target areas. For example, the target area includes four village areas. The facade image data of each single village and town building is extracted from the four village areas respectively, and the facade image data of one of the village areas can be used as a verification data set, and the facade image data of the remaining three village areas can be used as a training data set. The target recognition module Resnet50 combined with the target detector yolov5 model can be tested for generalization according to the above verification data set and training data set.

[0160] The step of extracting the facade image data of a plurality of single village and town buildings from the regional three-dimensional image is similar to operations S210-S250, and will not be repeated here.

[0161] The facade data of each single village and town building is input into the target recognition model, and the target recognition model can recognize the architectural features of the single village and town building.

[0162] In the embodiments of the present disclosure, the graphic output 550 can perform operations of: obtaining facade image data of a single village and town building and a plurality of labels associated with the single village and town building; and generating a visualization image according to the facade image data and the plurality of labels, the visualization image representing architectural features of the single village and town building.

[0163] For example, after recognizing the architectural features of the single village and town building, the recognition result can visually and associatively display the facade image data of the single village and town building and the corresponding plurality of labels. For example, on the visualization interface, after clicking the facade image data of the single village and town building in the architectural style classification, the detailed information of the architectural style of the single village and town building can be correspondingly displayed. Similarly, the detailed information of the architectural features of the single village and town building can also be associatively displayed.

[0164] For example, the recognition result can also visually and associatively display the architectural plan image and the plurality of labels of the single village and town building. The architectural plan image can be a plan view or a plan view.

[0165] Figure 5B A schematic diagram of the recognition result of the number of building stories according to the embodiments of the present disclosure is schematically shown. As shown in FIG. 5, the recognition result of the number of building stories of a plurality of single village and town buildings can also be visually displayed. Single village and town buildings with different numbers of building stories can be represented by different legends. The legends representing each single village and town building in the visualization image can be arranged according to the distribution of the corresponding each single village and town building in the target area. When clicking any legend in the visualization image, the related information of the single village and town building corresponding to the legend can be displayed, such as coordinate information and elevation information.

[0166] According to the embodiments of the present disclosure, the facade image data of the single village and town building and the plurality of labels are associatively displayed, which can directly and clearly show the plurality of architectural features of each single village and town building, so that the corresponding village and town planning strategy can be formulated according to the plurality of architectural features of each single village and town building in the region, and the utilization rate of the recognition result is improved. By visually displaying the recognition result and matching the visualization image with the data interaction interface, the demand for quickly querying the plurality of attribute features of the single building or the distribution of the same attribute feature village and town building in actual work can be met.

[0167] Based on the above-mentioned village and town building recognition method, the present disclosure also provides a village and town building recognition device. The following will be described in detail Figure 6 The device will be described in detail.

[0168] Figure 6A structural block diagram of the identification device of the village and town building is shown according to an embodiment of the present disclosure.

[0169] As shown in Figure 6 The identification device 600 of the village and town building of this embodiment includes a construction module 610, an extraction module 620, a labeling module 630, a generation module 640 and an identification module 650.

[0170] The construction module 610 is configured to construct a region three-dimensional image of a village and town building region based on sample oblique photography data, and the region three-dimensional image includes a plurality of single village and town building three-dimensional sub-images. In an embodiment, the construction module 610 can be configured to perform the operation S110 described above, and details are not repeated here.

[0171] The extraction module 620 is configured to extract facade image data of the plurality of single village and town building three-dimensional sub-images from the region three-dimensional image, respectively. In an embodiment, the extraction module 620 can be configured to perform the operation S120 described above, and details are not repeated here.

[0172] The labeling module 630 is configured to label a plurality of labels for each facade image data of the plurality of facade image data to obtain identification data of the plurality of single village and town building three-dimensional sub-images, and the plurality of labels respectively represent a plurality of types of building features of the single village and town building three-dimensional sub-images. In an embodiment, the labeling module 630 can be configured to perform the operation S130 described above, and details are not repeated here.

[0173] The generation module 640 is configured to generate a target identification model according to the identification data of the plurality of single village and town building three-dimensional sub-images. In an embodiment, the generation module 640 can be configured to perform the operation S140 described above, and details are not repeated here.

[0174] The identification module 650 is configured to identify the building features of the plurality of single village and town buildings in a target region by using the target identification model. In an embodiment, the identification module 650 can be configured to perform the operation S150 described above, and details are not repeated here.

[0175] According to an embodiment of the present disclosure, any of the modules of the constructing module 610, the extracting module 620, the labeling module 630, the generating module 640 and the identifying module 650 can be combined in one module, or any of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of the modules can be combined with at least part of the functions of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the constructing module 610, the extracting module 620, the labeling module 630, the generating module 640 and the identifying module 650 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of integrating or packaging a circuit, etc. or implemented by hardware or firmware, or implemented in any one of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the constructing module 610, the extracting module 620, the labeling module 630, the generating module 640 and the identifying module 650 can be at least partially implemented as a computer program module which can perform corresponding functions when the computer program module is run.

[0176] Figure 7 A block diagram of an electronic device suitable for implementing the method of identifying rural buildings according to an embodiment of the present disclosure is schematically shown.

[0177] As shown in Figure 7 The electronic device 700 according to an embodiment of the present disclosure includes a processor 701 which can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 702 or loaded from a storage portion 707 into a random access memory (RAM) 703. The processor 701 can include, for example, a general purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special purpose microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 701 can also include an on-board memory for cache use. The processor 701 can include a single processing unit or multiple processing units for performing different actions of the method processes according to embodiments of the present disclosure.

[0178] In the RAM 703, various programs and data required for the operation of the electronic device 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other via the bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 702 and / or the RAM 703. It should be noted that the programs can also be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0179] According to an embodiment of the present disclosure, the electronic device 700 can further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 can further include one or more of the following components connected to the I / O interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 708 including a hard disk, etc.; and a communication part 709 including a network interface card such as a LAN card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as necessary. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 710 as necessary, so that a computer program read out therefrom is installed in the storage part 708 as necessary.

[0180] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0181] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include but is not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), a portable compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703.

[0182] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the identification method of rural buildings or the automatic classification method of rural building properties based on three-dimensional remote sensing images provided by the embodiments of the present disclosure.

[0183] The above functions defined in the system / device of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. According to an embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0184] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of signals on a network medium, and be downloaded and installed through the communication part 709, and / or installed from the detachable medium 711. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to: wireless, wired, etc., or any appropriate combination thereof.

[0185] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the detachable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0186] According to embodiments of the present disclosure, program code of the computer program for performing the methods provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and can be implemented in a computer program product. Specifically, the computer program can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. The programming language includes, but is not limited to, Java, C++, python, “C” language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, and partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0187] The computer program product of the present disclosure can be a computer program product, which is a machine-readable medium (or computer readable medium) having stored therein a sequence of instructions executable by a machine such as a personal digital assistant (PDA), a laptop, a desktop computer, or a server. Alternatively, the computer program product can be a propagated signal per se generated by using the program code, and the program code can be stored on a machine-readable medium that can be read by a machine such as a personal digital assistant (PDA), a laptop, a desktop computer, or a server. The program code of the computer program product defines a function, preferably a corresponding action, and / or implements the elements / object / means for performing this function or a part thereof as disclosed in the embodiments of the present disclosure. The program code can be executed by using one or more processors.

[0188] Those skilled in the art will appreciate that features of the various embodiments and / or claims of the present disclosure can be combined or / and integrated with one another, even though such combinations or integrations are not expressly disclosed in the present disclosure. In particular, the features of the various embodiments and / or claims of the present disclosure can be combined and / or integrated with one another in any manner, without departing from the spirit and scope of the present disclosure. All such combinations and / or integrations are within the scope of the present disclosure.

[0189] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A method for identifying village and town buildings, comprising: Based on sample oblique photogrammetry data, a regional three-dimensional image of the village and town building area is constructed, and the regional three-dimensional image includes multiple individual village and town building three-dimensional sub-images; Elevation image data of the multiple individual village and town buildings in the three-dimensional image of the region are extracted respectively. Multiple labels are assigned to each of the multiple facade image data to obtain the identification data of the multiple individual village and town building three-dimensional sub-images. The multiple labels respectively characterize the architectural features of multiple types of individual village and town building three-dimensional sub-images. Based on the identification data of the multiple individual village and town building 3D sub-images, a target recognition model is generated; as well as Using the target recognition model, the architectural features of multiple individual village and town buildings within the target area are identified.

2. The identification method according to claim 1, wherein, The step of extracting facade image data of the multiple individual village and town building 3D sub-images from the 3D image of the region includes: Obtain the 3D point cloud data and top view image of the region; The top-view image is projected onto the three-dimensional point cloud data to filter out the first point cloud data and the second point cloud data of the multiple individual village and town building three-dimensional sub-images from the three-dimensional point cloud data. The first point cloud data represents the boundary of the individual village and town building three-dimensional sub-image, and the second point cloud data represents the texture of the individual village and town building three-dimensional sub-image. Based on the first point cloud data, the spatial structure of the multiple individual village and town building three-dimensional sub-images is obtained by segmenting from the three-dimensional image of the region; Based on the second point cloud data, texture mapping is performed on the spatial structure of the multiple individual village and town building 3D sub-images to obtain the multiple individual village and town building 3D sub-images; and facade image data of the multiple individual village and town building 3D sub-images are extracted respectively.

3. The identification method according to claim 1, wherein, The step of labeling each facade image data with multiple tags to obtain the identification data of the multiple individual village and town building 3D sub-images includes: Multiple annotation models are used to annotate each facade data of the facade image data with multiple labels, and the multiple annotation models are used to annotate multiple types of labels respectively; The labeled facade image data is cross-validated using the multiple annotation models; and If the verification of the labeled facade image data is successful, multiple discrete variables are set for each facade image data according to the multiple labels to obtain the identification data of the multiple individual village and town building three-dimensional sub-images.

4. The identification method according to claim 1, wherein, The identification data of the multiple individual village and town building 3D sub-images includes training set data, validation set data, and test set data; The step of generating a target recognition model based on the identifier data of the multiple individual village and town building 3D sub-images includes: For each of the multiple candidate recognition models, perform the following operations to obtain multiple test results: The candidate recognition model is trained using the training set data to obtain the trained candidate recognition model; The trained candidate recognition model is validated using the validation set data to obtain validation results; and If the verification result is found to be passed, the trained candidate recognition model is tested using the test set data to obtain the test result for the candidate recognition model.

5. The identification method according to claim 4, wherein, The step of generating a target recognition model based on the identifier data of the multiple individual village and town building 3D sub-images further includes: The target recognition model is determined from the multiple candidate recognition models based on the optimal test result among the multiple test results.

6. The identification method according to claim 4 or 5, wherein, The step of generating a target recognition model based on the identifier data of the multiple individual village and town building 3D sub-images further includes: Using a target detector, door data, window data, and facade image data of the multiple individual village and town buildings are extracted from the multiple 3D sub-images, respectively; and By combining the multiple labels and using the door data, window data, and facade image data, the target recognition model is iteratively trained to obtain an optimized target recognition model.

7. The identification method according to claim 6, wherein, The target detector includes a convolutional layer, a target detection layer, and a filtering layer. The target detector extracts door data, window data, and facade image data of multiple individual village and town buildings from multiple 3D sub-images, including: The convolutional layer extracts multiple initial feature data from multiple three-dimensional sub-images; The target detection layer performs feature detection on the multiple initial feature data to obtain initial door data, initial window data, and initial facade image data; and The filtering layer filters out the door data, window data, and facade image data from the initial door data, initial window data, and initial facade image data, respectively.

8. The identification method according to claim 1, wherein, The process of using the target recognition model to identify individual village and town buildings within the target area includes: Obtain oblique photographic data of the target area; Based on the oblique photographic data of the target area, a three-dimensional image of the target area is constructed; Extract facade image data of multiple individual village and town buildings from the three-dimensional image of the region; and The facade data is input into the target recognition model to obtain the architectural characteristics of multiple individual village and town buildings within the target area.

9. The identification method according to claim 1 further includes: Obtain the facade image data of the individual village / town building and multiple tags associated with the individual village / town building; as well as Based on the facade image data and the multiple labels, a visualization image is generated, which represents the architectural features of the individual village buildings.

10. A device for identifying village and town buildings, comprising: The construction module is used to construct a regional three-dimensional image of the village and town building area based on sample oblique photogrammetry data. The regional three-dimensional image includes multiple individual village and town building three-dimensional sub-images. The extraction module is used to extract the facade image data of the multiple individual village and town building three-dimensional sub-images from the three-dimensional image of the region, respectively; The annotation module is used to annotate each of the multiple facade image data with multiple labels to obtain the identification data of the multiple individual village and town building three-dimensional sub-images. The multiple labels respectively represent the architectural characteristics of multiple types of the individual village and town building three-dimensional sub-images. The generation module is used to generate a target recognition model based on the identification data of the multiple individual village and town building 3D sub-images; as well as The identification module is used to identify the architectural features of multiple individual village and town buildings within the target area using the target identification model.

11. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 9.

Images