A method and system for extracting glass area on building facade

Abstract

The present invention provides a method and system for extracting glass areas on building facades, the method comprising: extracting glass areas from multi-view oblique photography images based on a deep learning algorithm; extracting facade point cloud data from a three-dimensional building model; extracting the building facade glass area from the facade point cloud data according to the mapping relationship between the three-dimensional building model and the oblique photography image and the glass area extracted from the oblique photography image. The method of the present invention focuses on the demand for automatic modeling of oblique photogrammetry, and for the building facade glass in the city, it utilizes the dense matching point cloud facade information generated by multi-view images and the spatial structure information of the target in the oblique image, and can effectively extract the semantic information of the facade glass area, providing basic data support for the city's architectural semantic understanding and building model optimization.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to a method and system for extracting a building facade glass area. Background Art

[0002] With the rapid development of 3D modeling algorithms and hardware, real-life 3D models of cities are widely used in construction planning, autonomous driving, emergency management and other fields. Among them, 3D modeling technology based on oblique photogrammetry of unmanned aerial vehicles (UAVs) is highly efficient and low-cost, and is very suitable for large-scale urban scene reconstruction. At the same time, with the process of urbanization, buildings with facade glass areas are becoming more and more common, such as glass walls, glass doors, glass balconies, etc. Due to the special properties of glass materials and the limitations of feature detection and matching algorithms, the reconstruction quality of the facade glass area of ​​buildings is poor, with varying degrees of geometric deformation, texture blur and other problems. Extracting the facade glass area of ​​a building can not only provide important information for applications such as urban lighting analysis, light pollution assessment, electrical signal processing, energy conservation and emission reduction, but also assist in the repair of the facade glass area of ​​the building model, thereby beautifying the model.

[0003] Extracting glass areas from building facades in oblique photography images requires the use of high-resolution oblique images and the spatial context features they contain. On the other hand, existing glass detection research mainly focuses on glass in natural images to provide navigation information for intelligent robots. Compared with glass in natural images, glass in oblique images has greater scale variations and more appearance types (transparent, colored, or mirror-like facade glass areas). In addition, due to complex shooting conditions and high imaging angles, glass in oblique images often has large distortions. Therefore, detecting glass from oblique images is more challenging. Summary of the invention

[0004] The present invention aims at the technical problems existing in the prior art and provides a method and system for extracting glass areas on building facades.

[0005] According to a first aspect of the present invention, a method for extracting glass areas on building facades is provided, comprising: extracting glass areas from multi-view oblique photographic images based on a deep learning algorithm; extracting facade point cloud data from a three-dimensional model of the building; and extracting the glass areas on the building facade from the facade point cloud data according to a mapping relationship between the three-dimensional model of the building and the oblique photographic images and the glass areas extracted from the oblique photographic images.

[0006] Based on the above technical solution, the present invention can also make the following improvements.

[0007] Optionally, the method of extracting glass areas from multi-view oblique photographic images based on a deep learning algorithm includes: selecting multiple oblique photographic images in which facade glass areas exist, and marking each glass area in each oblique photographic image to form training samples; training a preset deep learning network model based on the training samples to obtain a trained deep learning network model; and extracting glass areas from multi-view oblique photographic images based on the trained deep learning network model, wherein the glass areas include multiple glass sub-areas in the oblique photographic images, wherein the multiple glass sub-areas have a spatial position relationship.

[0008] Optionally, extracting the facade point cloud data from the three-dimensional building model includes: tilting the point cloud P in the model i Project it onto the plane where the ground is located and get the projected point p i ; Record each projection point p i point cloud density and elevation difference; based on the set point cloud density threshold and elevation difference threshold, extract the facade point cloud data from the building 3D model.

[0009] Optionally, the recording of each projection point p i Point cloud density and elevation difference, including: recording point cloud P i and the corresponding projection point p i The Euclidean distance between them is recorded on the density map as the elevation difference, and the projection point p i The number of adjacent points within the set domain is taken as the point cloud density; accordingly, based on the set point cloud density threshold and elevation difference threshold, the facade point cloud data is extracted from the three-dimensional model of the building, including: extracting point clouds with a point cloud density greater than the set point cloud density threshold and an elevation difference greater than the set elevation difference threshold as the facade point cloud data, and forming a facade triangle face set T set .

[0010] Optionally, extracting the building facade glass area from the facade point cloud data according to the mapping relationship between the building three-dimensional model and the oblique photography image and the glass area extracted from the oblique photography image includes: extracting the building facade glass area from the facade triangle facet set T set Filter out visible triangles T i ; The visible triangle T i , projected into the oblique photography image, denoted as S i , if S i If two or more points of S fall into the glass area extracted from the oblique photography image, then S i is a candidate triangle containing glass; for each visible triangle T i Make a judgment and form a candidate set of triangle faces containing glass F set ; For the candidate triangle face set F containing glassset Each triangular face in the image is screened again to determine the final building facade glass area.

[0011] Optionally, the set of triangle patches T from the vertical face set Filter out visible triangles T i , including: opposite triangle face set T set The triangle patch T i , whose normal vector is Center is C i , remember C i To camera center O i The vector is calculate and If the angle between them is less than 90°, the triangle T i The visible triangles.

[0012] Optionally, the visible triangle T i , projected onto the oblique photography image, including: according to the pixel plane coordinate system (u, v) and the model coordinate system (X w , Y w , Z w ) mapping relationship, the visible triangle patch T i , projected onto the oblique photography image; where the pixel plane coordinate system (u, v) and the model coordinate system (X w , Y w , Z w ) is:

[0013]

[0014] Among them, dx and dy represent the horizontal and vertical dimensions of a single pixel respectively; u0 and v0 represent the horizontal and vertical coordinates of the image principal point in the image plane coordinate system respectively; f represents the focal length of the camera; R is the 3×3 rotation matrix composed of the camera's external orientation elements, 0 T is (0, 0, 0); t is the translation vector of the camera optical center in the world coordinate system; X w , Y w , Z w is the model coordinate value.

[0015] Optionally, the candidate set of triangle facets F containing glass set Each triangle in the set F is screened again to determine the final building facade glass area, including: for the candidate triangle set F containing glass setFor two triangular facets with high overlap, based on the imaging principle, the distance from each triangular facet to the camera center is calculated, and the triangular facets close to the camera center are retained; for the retained triangular facets, isolated triangular facets are deleted to obtain the final building facade glass area.

[0016] According to a second aspect of the present invention, a building facade glass area extraction system is provided, comprising: a first extraction module for extracting glass areas from multi-view oblique photography images based on a deep learning algorithm; a second extraction module for extracting facade point cloud data from a three-dimensional model of the building; and a third extraction module for extracting the building facade glass area from the facade point cloud data based on a mapping relationship between the three-dimensional model of the building and the oblique photography image and the glass area extracted from the oblique photography image.

[0017] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein the processor is configured to implement the steps of a method for extracting a building facade glass area when executing a computer management program stored in the memory.

[0018] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium on which a computer management program is stored, and when the computer management program is executed by a processor, the steps of the method for extracting a building facade glass area are implemented.

[0019] The present invention provides a method and system for extracting building facade glass areas, focusing on the needs of automatic modeling of oblique photogrammetry. For urban building facade glass, the dense matching point cloud facade information generated by multi-view images and the spatial structure information of targets in oblique images are utilized, and the semantic information of the facade glass area can be effectively extracted, providing basic data support for urban architectural semantic understanding and building model optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flow chart of a building facade glass area extraction method provided by the present invention;

[0021] Figure 2 Schematic diagram of the method for extracting glass from oblique images;

[0022] Figure 3 It is a schematic diagram of the method for extracting elevation information from point clouds;

[0023] Figure 4 Schematic diagram of the method for establishing the correspondence between triangles and images;

[0024] Figure 5 An overall flow chart of a building facade glass area extraction method provided by the present invention;

[0025] Figure 6A structural schematic diagram of a building facade glass area extraction system provided by the present invention;

[0026] Figure 7 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0027] Figure 8 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0028] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0029] The semantic information of the building facade glass area provides an important basis for applications such as urban lighting analysis, light pollution assessment, electrical signal processing, energy conservation and emission reduction, and can assist in model repair and beautify the three-dimensional model. The present invention proposes a method for detecting building facade glass from multi-view oblique photography to provide support for the application of urban real-scene models.

[0030] The core idea of ​​the method of the present invention is to utilize the rich contextual features and high resolution of oblique photography images, combine them with the geometric structure information (facade information) of point clouds, and extract the facade glass area of ​​the building based on the correspondence between the image and the triangular facets under the multi-view constraints of the oblique photography images.

[0031] Embodiment 1

[0032] A method for extracting glass areas from building facades, see Figure 1 , the facade glass area extraction method mainly includes the following steps:

[0033] S1, extracts glass areas from multi-view oblique photography images based on deep learning algorithms.

[0034] As an embodiment, the method of extracting a glass area from a multi-view oblique photographic image based on a deep learning algorithm includes: selecting multiple oblique photographic images in which a facade glass area exists, and marking each glass area in each oblique photographic image to form a training sample; training a preset deep learning network model based on the training samples to obtain a trained deep learning network model; extracting a glass area from the multi-view oblique photographic image based on the trained deep learning network model, the glass area including multiple glass sub-areas in the oblique photographic image, wherein the multiple glass sub-areas have a spatial position relationship.

[0035] It can be understood that in the embodiment of the present invention, the glass area is extracted from the multi-view oblique photography images based on the deep learning network. Figure 2Before using the deep learning network model to extract the glass area, the preset deep learning network model needs to be trained. First, prepare the training sample data: detectors based on deep learning algorithms often need to learn a large amount of data. Select the oblique image with the facade glass area, crop it to 512×512 pixels, and manually mark each glass area in the image. Different from the conventional instance segmentation data annotation, the method of the embodiment of the present invention takes into account the spatial relationship between the glass blocks. This strategy can effectively improve the performance of the detector. In addition, in order to improve the generalization ability of the model and obtain more learning samples, the data is enhanced and expanded, including rotation, mirroring, Gamma transformation, etc.

[0036] After the training sample data is prepared, the deep learning network model is trained based on the training sample data. Specifically, the deep learning framework Pytorch is used for training under the Ubuntu 16.04 operating system, and the deep learning network model is trained as the ImageNet pre-trained model. During the training process, an 11G RTX2080Ti graphics card is used and CUDA is used for acceleration. Some important parameters are batch size (batch size) of 2, initial learning rate of 0.002, and decreased at 60000, 80000, and 100000 iterations respectively, weight decay and momentum are 0.0001 and 0.9 respectively, and the model is trained 120000 times, and the loss no longer decreases.

[0037] The trained model is applied to the multi-view oblique image to obtain the pixel-by-pixel detection results (binary image) of the glass in the oblique image. In the detection results, the pixel value of the glass area is 1 (mask i ), the pixel value of the non-glass area is 0.

[0038] S2, extracts facade point cloud data from the building 3D model.

[0039] As an embodiment, the step of extracting facade point cloud data from a three-dimensional building model includes:

[0040] The point cloud P in the tilted model i Project it onto the plane where the ground is located and get the projected point p i ;

[0041] Record each projection point p i point cloud density and elevation difference; based on the set point cloud density threshold and elevation difference threshold, extract the facade point cloud data from the building 3D model.

[0042] Wherein, the recording of each projection point p i Point cloud density and elevation difference, including: recording point cloud P iand the corresponding projection point p i The Euclidean distance between them is recorded on the density map as the elevation difference, and the projection point p i The number of adjacent points within the set domain is taken as the point cloud density; accordingly, based on the set point cloud density threshold and elevation difference threshold, the facade point cloud data is extracted from the three-dimensional model of the building, including: extracting point clouds with a point cloud density greater than the set point cloud density threshold and an elevation difference greater than the set elevation difference threshold as the facade point cloud data, and forming a facade triangle face set T set .

[0043] Specifically, the method for extracting facade point cloud data from a three-dimensional building tilt model can be found in Figure 3 ,

[0044] First, the tilted model point cloud (P i ) is projected onto the plane where the ground is located (Ax+By+Cz=D), and the projected point (p i ), and record P i With p i The Euclidean distance between them is the elevation difference HG i . Secondly, calculate the projected point (p i ) The number of adjacent points (PD) in the density map within the domain (R) region ) as a point (p i ) point cloud density, and the maximum elevation difference (HG region ). Finally, the point cloud density threshold (V1) and elevation difference threshold (V2) are set based on experience, and the point cloud is screened by binary classification to extract the building's facade point cloud data. It should be noted that this step does not require high-precision results. The purpose is to filter most of the non-facade point clouds, reduce the error of the oblique image detection results, and improve the calculation efficiency. The set of extracted facade triangles is denoted as T set .

[0045] S3, extracting the building facade glass area from the facade point cloud data according to the mapping relationship between the building three-dimensional model and the oblique photography image and the glass area extracted from the oblique photography image.

[0046] As an embodiment, according to the mapping relationship between the three-dimensional building model and the oblique photography image and the glass area extracted from the oblique photography image, extracting the building facade glass area from the facade point cloud data includes: extracting the building facade glass area from the facade triangle face set T set Filter out visible triangles T i ; The visible triangle T i , projected into the oblique photography image, denoted as S i , if S iIf two or more points of S fall into the glass area extracted from the oblique photography image, then S i is a candidate triangle containing glass; for each visible triangle T i Make a judgment and form a candidate set of triangle faces containing glass F set ; For the candidate triangle face set F containing glass set Each triangular face in the image is screened again to determine the final building facade glass area.

[0047] Understandably, see Figure 4 , based on multi-view constraints, the correspondence between triangular facets and oblique images is established to extract the glass area of ​​the building facade, which specifically includes the following steps:

[0048] (1) Visual triangle patch judgment. Opposite triangle patch T set The triangle patch T i , whose normal vector is Center is C i , remember C i To camera center O i The vector is calculate and If the angle is less than 90°, the triangle is a visible triangle, and (2) is executed. Otherwise, the next triangle T is determined. j .

[0049] (2) Projection judgment. The pixel plane coordinate system (u, v) and the model coordinate system (X w , Y w , Z w ) The mapping relationship can be expressed by formula (1). Where dx and dy represent the horizontal and vertical dimensions of a single pixel, respectively; u0 and v0 represent the horizontal and vertical coordinates of the principal point in the image plane coordinate system, respectively; f represents the focal length of the camera; R is the 3×3 rotation matrix composed of the camera's external orientation elements, 0 T is (0, 0, 0); t is the translation vector of the camera optical center in the world coordinate system; X w , Y w , Z w is the model coordinate value.

[0050] The visual triangle T i , projected into the oblique image and recorded as S i , combined with the glass detection results of the oblique image (mask i ), if S i Two or more points of the mask fall within i In this paper, we believe that S iIt is very likely to be glass, otherwise it is not, and the set of triangular facets that are very likely to be glass is recorded as F set .

[0051]

[0052] (3) Result screening. set The triangles in the image may overlap. For two triangles with high overlap, the distance between the triangle and the camera center is used for screening. Specifically, according to the imaging principle, the triangle close to the camera center is the correctly projected triangle, and vice versa, it is the incorrectly projected triangle.

[0053] (4) Result optimization: For the correctly projected triangular facets in (3), the isolated triangular facets are deleted to obtain the final building facade glass area extraction result.

[0054] Embodiment 2

[0055] A method for extracting glass areas from building facades, see Figure 5 , the basic steps of this extraction method include:

[0056] (1) Glass detection in oblique images: This method is based on a deep learning algorithm that integrates an instance segmentation network and an edge feature extraction network. Specifically, the instance segmentation module considers the spatial relationship between glass blocks to improve the detection results, and uses edge features to further optimize the segmentation accuracy of glass edges.

[0057] (2) Extraction of facade information from point clouds: Building facades are generally perpendicular to the ground and have large elevation differences. Multi-view images are used to generate dense point clouds through image matching algorithms. The point clouds are projected onto the ground to generate point cloud density maps and point cloud height difference maps. Thresholds are set based on empirical values ​​to divide the point clouds into facade point clouds and non-facade point clouds. In theory, the facade area has greater point cloud density and height difference. This method can filter out most non-facade point clouds.

[0058] (3) Extracting glass from the 3D model of the building. Based on the mapping relationship between the 3D model and the oblique image, under the multi-view constraint, establish the correspondence between the facade triangles obtained in step (2) and the oblique image. Based on the oblique image glass detection results, extract the building's facade glass area.

[0059] Embodiment 3

[0060] A building facade glass extraction system, see Figure 6 The extraction system includes a first extraction module 601, a second extraction module 602 and a third extraction module 603.

[0061] Among them, the first extraction module 601 is used to extract the glass area from the multi-view oblique photography image based on the deep learning algorithm; the second extraction module 602 is used to extract the facade point cloud data from the three-dimensional model of the building; the third extraction module 603 is used to extract the building facade glass area from the facade point cloud data according to the mapping relationship between the three-dimensional model of the building and the oblique photography image and the glass area extracted from the oblique photography image.

[0062] It can be understood that the building facade glass area extraction system provided by the present invention corresponds to the building facade glass area extraction method provided by the aforementioned embodiments. The relevant technical features of the building facade glass area extraction system can refer to the relevant technical features of the building facade glass area extraction method, which will not be repeated here.

[0063] Embodiment 4

[0064] See also Figure 7 , Figure 7 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 7 As shown, an embodiment of the present invention provides an electronic device 700, including a memory 710, a processor 720, and a computer program 711 stored in the memory 710 and executable on the processor 720. When the processor 720 executes the computer program 711, the method for extracting the building facade glass area of ​​embodiment one or embodiment two is implemented.

[0065] Embodiment 5

[0066] See also Figure 8 , Figure 8 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 8 As shown, this embodiment provides a computer-readable storage medium 800 on which a computer program 811 is stored. When the computer program 811 is executed by a processor, the building facade glass extraction method of the first or second embodiment is implemented.

[0067] The present invention provides a method and system for extracting building facade glass areas, which have the following advantages:

[0068] (1) The present invention proposes a detection method that combines the spatial structure relationship and edge features of glass blocks, which can better extract glass in oblique images. On the one hand, the spatial structure relationship can avoid missing small glass blocks and improve the detection recall rate; on the other hand, the edge features can make up for the shortcomings of instance segmentation in object edge segmentation and improve segmentation accuracy.

[0069] (2) The present invention proposes to use the projection point cloud density and elevation difference to effectively filter out non-facade point clouds, further ensuring the accuracy of facade glass area extraction and improving calculation efficiency;

[0070] (3) The facade glass area extraction method based on the geometric structure (facade information) of the point cloud and multi-view constraints proposed in the present invention can effectively extract the facade glass area of ​​the inclined model building. On the one hand, the facade information can reduce the error introduced by the image misdetection, thereby ensuring the reliability of the method. On the other hand, the multi-view constraints further optimize the facade glass area extraction accuracy.

[0071] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0072] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0076] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0077] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for extracting glass areas from building facades, characterized in that: include: Based on deep learning algorithms, glass areas are extracted from multi-view oblique photography images; Extract the facade point cloud data from the 3D model of the building, and all the facade point cloud data constitute a set of facade triangles ; Extracting the building facade glass area from the facade point cloud data according to the mapping relationship between the building three-dimensional model and the oblique photography image and the glass area extracted from the oblique photography image; The step of extracting the building facade glass area from the facade point cloud data according to the mapping relationship between the building three-dimensional model and the oblique photography image and the glass area extracted from the oblique photography image comprises: From the facade triangle patch collection Filter out visible triangles ; The visible triangle , projected onto the oblique photography image, recorded as ,if If two or more points of the image fall into the glass area extracted from the oblique photography image, then The candidate triangles containing glass; For each visible triangle Make a judgment and form a set of candidate triangles containing glass ; For the candidate triangle face set containing glass Each triangular face in the image is screened again to determine the final building facade glass area.

2. The building facade glass area extraction method according to claim 1, characterized in that: The method of extracting the glass area from the multi-view oblique photography images based on the deep learning algorithm includes: Select multiple oblique photography images with facade glass areas, and mark each glass area in each oblique photography image to form training samples; Training a preset deep learning network model based on the training samples to obtain a trained deep learning network model; Based on the trained deep learning network model, a glass area is extracted from the multi-view oblique photography image, wherein the glass area includes a plurality of glass sub-areas in the oblique photography image, wherein the plurality of glass sub-areas have a spatial position relationship.

3. The method for extracting building facade glass areas according to claim 1, characterized in that: The step of extracting the facade point cloud data from the three-dimensional building model comprises: Tilt the point cloud in the model Project it onto the plane where the ground is located to get the projected point ; Record each projection point Point cloud density and elevation difference; Based on the set point cloud density threshold and elevation difference threshold, the facade point cloud data is extracted from the 3D model of the building.

4. The building facade glass area extraction method according to claim 3, characterized in that: The recording of each projection point Point cloud density and elevation difference, including: Recording point cloud and the corresponding projection point The Euclidean distance between them is recorded as the elevation difference on the density map, and the projection point The number of adjacent points within the set area is taken as the point cloud density; Accordingly, based on the set point cloud density threshold and elevation difference threshold, the facade point cloud data is extracted from the building 3D model, including: Extract point clouds whose point cloud density is greater than the set point cloud density threshold and whose elevation difference is greater than the set elevation difference threshold as the facade point cloud data, forming a facade triangle facet set .

5. The method for extracting building facade glass areas according to claim 1, characterized in that: The triangle face set from the vertical face Filter out visible triangles ,include: Opposite triangle patch set The triangles in , whose normal vector is , the center is , remember To the center of the camera The vector is ,calculate and If the angle between is less than 90°, the triangle The visible triangles.

6. The method for extracting building facade glass areas according to claim 1, characterized in that: The visible triangle , projected onto oblique photography images, including: According to the pixel plane coordinate system ( u , v ) and the model coordinate system ( ) mapping relationship, the visible triangle , projected into the oblique photography image; Among them, the pixel plane coordinate system ( u , v ) and the model coordinate system ( ) is: ； in, , Represent the horizontal and vertical dimensions of a single pixel respectively; , Respectively represent the abscissa and ordinate of the image principal point in the image plane coordinate system; Represents the focal length of the camera; is the 3×3 rotation matrix consisting of the camera's exterior orientation elements, is (0, 0, 0); t is the translation vector of the camera optical center in the world coordinate system; , , is the model coordinate value.

7. The method for extracting building facade glass areas according to claim 1, characterized in that: The candidate pair of triangle facets containing glass Each triangular face in the image is screened again to determine the final building facade glass area, including: For the candidate set of triangles containing glass For two triangles with high overlap, based on the imaging principle, the distance from each triangle to the camera center is calculated, and the triangle closest to the camera center is retained; For the retained triangular faces, isolated triangular faces are deleted to obtain the final building facade glass area.

8. A building facade glass area extraction system, characterized in that: include: A first extraction module is used to extract glass areas from multi-view oblique photography images based on a deep learning algorithm; The second extraction module is used to extract the facade point cloud data from the building 3D model. All the facade point cloud data constitute a facade triangle facet set. ; A third extraction module is used to extract the building facade glass area from the facade point cloud data according to the mapping relationship between the building three-dimensional model and the oblique photography image and the glass area extracted from the oblique photography image; The step of extracting the building facade glass area from the facade point cloud data according to the mapping relationship between the building three-dimensional model and the oblique photography image and the glass area extracted from the oblique photography image comprises: From the facade triangle patch collection Filter out visible triangles ; The visible triangle , projected onto the oblique photography image, recorded as ,if If two or more points of the image fall into the glass area extracted from the oblique photography image, then The candidate triangles containing glass; For each visible triangle Make a judgment and form a set of candidate triangles containing glass ; For the candidate triangle face set containing glass Each triangular face in the image is screened again to determine the final building facade glass area.

9. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by a processor, the steps of the building facade glass area extraction method according to any one of claims 1 to 7 are implemented.

Images