Semantic completion method and system suitable for building facade image

Abstract

The invention solves the technical problem of how to complement the semantic information extracted from the building facade image, and provides a semantic complementation method and system suitable for the building facade image based on progressive contour correction and size unification processing. According to the method, a building monomer model facade texture-semantic label data set is utilized, iteration processing is carried out through rectangular contour completion, missing or redundant parts possibly existing in building facade semantic information are completed or cut, unification processing is carried out on windows in the same row, and semantic information with better integrity and normalization is obtained. Compared with the prior art, the method has the advantages that the semantic information of the existing building facade can be standardized and complemented in combination with the structural characteristics of the building facade, and the high-quality semantic information which can be used for synthesizing the general texture of the building facade can be obtained. Therefore, the method has important practical value and wide application prospect.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision semantic completion, and mainly proposes a building facade image semantic completion method based on progressive contour correction and size unification, which will be applied to related fields such as building three-dimensional reconstruction and digital city construction. Background Art

[0002] The single-body model of a building has rich topological semantic information, but its texture mapping results are often occluded due to the proximity of the spatial position of the objects. There are inevitable missing parts in the facade texture, which affects the visual effect of the model. How to use the unique structural features of the building facade texture to complete the extracted semantic information is the key to improving the visual quality and accuracy of the single-body model of the building. Traditional image restoration often ignores the structural characteristics of the building. Deep learning-based image completion algorithms usually use bounding boxes or key points for constraints, which are not effective in complex scenes and still have occlusion problems. Summary of the invention

[0003] The technical problem solved by the present invention is how to complete the semantic information extracted from the building facade image. Based on progressive contour correction and size unification processing, a semantic completion method suitable for building facade images is provided. The present invention uses the building monomer model facade texture-semantic label dataset, performs iterative processing through rectangular contour completion, completes or cuts the missing or redundant parts of the building facade semantic information, and unifies the windows in the same row to obtain semantic information with better integrity and standardization.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a method for semantic completion of building facade images based on progressive contour correction and size unification processing, the method comprising the following steps:

[0005] Step 1, data preparation: at least including a building facade texture image containing windows and a corresponding semantic label, the semantic label at least including semantic information of the background and the window;

[0006] Step 2, using rectangular outline completion processing to complete the semantic information of the window;

[0007] Step 3, repeating the operation of step 2, and continuously performing rectangular contour completion processing on the semantic information of the window according to the ratio of the number of window pixels to the number of non-window pixels in the semantic information, until the semantic information of the window meets the requirements;

[0008] Step 4: Selectively resize the windows in the same row to obtain the complete semantic label of the building facade texture image.

[0009] Furthermore, the semantic labels in step 1 are obtained through the building facade image semantic extraction model.

[0010] Furthermore, in step 2, the specific method of the rectangular outline completion process is as follows:

[0011] Step 2.1, extract the maximum bounding rectangle R for each window in the semantic label;

[0012] Step 2.2, rectangular semantic pixel ratio threshold determination: Calculate the window pixel N in the maximum bounding rectangle R 1 and the number of non-window pixels N 2 , calculate the ratio of non-window pixels to window pixels, rat, where rat = N 2 / N 1 ; If rat is less than the set threshold t, the rectangle is the window range; otherwise proceed to the next step;

[0013] Step 2.3, rectangle boundary point transformation calculation: Get the coordinates of the upper left point of the maximum circumscribed rectangle in step 2.2 (x 1 ,y 1 ) and the lower right corner coordinate (x 2 ,y 2 ), set the boundary point transformation coefficient Δ 1 , Δ 2 With the constant factor l, Δ 1 / (x 2 -x 1 )=Δ 2 / (y 2 -y 1 )=l, calculate the coordinates (x 1 +Δ 1 ,y 1 +Δ 2 ) and (x 2 -Δ 1 ,y 2 -Δ 2 ), where (Δ 1 / Δ 2 )=(x 2 -x 1 ) / (y 2 -y 1 );

[0014] Step 2.4, calculation of the semantic proportion of the directional rectangle: take the four sides of the largest circumscribed rectangle as the length, and the boundary point transformation coefficient Δ 1 With Δ 2 is the width, set the left, right, top and bottom four rectangles, the area is S 1 , S 2 , S 3 , S 4; where S 1 =S 2 =(y 2 -y 1 )×Δ 1 , S 3 =S 4 =(x 2 -x 1 )×Δ 2 ; Calculate the number of window pixels in each rectangle and find its ratio to the total number of pixels in the rectangle r 1 、r 2 、r 3 、r 4 ;

[0015] Step 2.5, scale decision and rectangle correction: If r 1 Minimum, then (x 1 +Δ 1 ,y 1 ) and (x 2 ,y 2 ) is the diagonal line to make a new rectangle R ′ ; if r 2 Minimum, then (x 1 ,y 1 ) and (x 2 -Δ 1 ,y 2 ) is a diagonal; if r 3 Minimum, then (x 1 ,y 1 +Δ 2 ) and (x 2 ,y 2 ) is a diagonal; if r 4 Minimum, then (x 1 ,y 1 ) and (x 2 ,y 2 -Δ 2 ) is a diagonal line, and the new rectangle is the semantic information range of the current window.

[0016] Furthermore, in step 3, the specific method of progressive contour correction is as follows:

[0017] Step 3.1: New rectangle semantic pixel ratio threshold determination: For the new rectangle R obtained in step 2 ′ , calculate the number of pixels N in the window 1 ′ The number of pixels N in the non-window ′ 2 , calculate rat ′ =N ′ 2 / N ′1 ; if rat ′ is less than the threshold t, then the new rectangle R ′ is the window range; otherwise proceed to the next step;

[0018] Step 3.2, rectangle contour completion process iteration: transform the new rectangle R ′ Instead of the maximum circumscribed rectangle R in step 2.3, perform the operations of steps 2.3, 2.4, and 2.5 until the ratio of the number of non-window pixels to the number of window pixels in the obtained rectangle is less than the threshold t. In this case, the rectangle is the corresponding range of the window and the iteration terminates.

[0019] Furthermore, in step 4, the specific method of unifying the sizes of windows in the same row is as follows:

[0020] Step 4.1, calculation of semantic information Z_Score of windows in the same row: For windows in the same row, calculate the mean Where N is the number of windows in the row, i is the window number, and X i is the number of pixels in the window; calculate the standard deviation And use Calculate the Z_Score of each window in this row;

[0021] Step 4.2, rectangle range optimization based on Z_Score: For the semantic information of windows in the same row, if the Z_Score of the window is greater than m or less than -m, where m is a constant, its rectangle range is retained without change; the area average of the remaining windows is calculated again, and a window is found among them so that its semantic information rectangle area is closest to the area average, and the semantic information range of the remaining windows in the same row is constrained to this rectangle;

[0022] Step 4.3, row-by-row semantic label optimization: Repeat steps 4.1 and 4.2 for each row of windows to finally obtain the complete semantic label of the building facade texture image.

[0023] The present invention also provides a semantic completion system applicable to building facade images, comprising the following modules:

[0024] A data preparation module, used for preparing data, wherein the data at least includes a texture image of a building facade containing windows and a corresponding semantic label, wherein the semantic label at least includes semantic information of a background and a window;

[0025] A rectangular outline completion processing module is used to complete the semantic information of the window by using the rectangular outline completion processing;

[0026] A repeating processing module is used to repeat the operation of the rectangular outline completion processing module, and continuously perform rectangular outline completion processing on the semantic information of the window according to the ratio of the number of window pixels to the number of non-window pixels in the semantic information, until the semantic information of the window meets the requirements;

[0027] The size unification processing module is used to selectively unify the size of windows in the same row, and finally obtain the complete semantic label of the building facade texture image.

[0028] Furthermore, the semantic labels in the data preparation module are obtained through the building facade image semantic extraction model.

[0029] Furthermore, in the rectangular contour completion processing module, the specific method of the rectangular contour completion processing is as follows:

[0030] Step 2.1, extract the maximum bounding rectangle R for each window in the semantic label;

[0031] Step 2.2, rectangular semantic pixel ratio threshold determination: Calculate the window pixel N in the maximum bounding rectangle R 1 and the number of non-window pixels N 2 , calculate the ratio of non-window pixels to window pixels, rat, where rat = N 2 / N 1 ; If rat is less than the set threshold t, the rectangle is the window range; otherwise proceed to the next step;

[0032] Step 2.3, rectangle boundary point transformation calculation: Get the coordinates of the upper left point of the maximum circumscribed rectangle in step 2.2 (x 1 ,y 1 ) and the lower right corner coordinate (x 2 ,y 2 ), set the boundary point transformation coefficient Δ 1 , Δ 2 With the constant factor l, Δ 1 / (x 2 -x 1 )=Δ 2 / (y 2 -y 1 )=l, calculate the coordinates (x 1 +Δ 1 ,y 1 +Δ 2 ) and (x 2 -Δ 1 ,y 2 -Δ 2 ), where (Δ 1 / Δ 2 )=(x 2 -x 1 ) / (y2 -y 1 );

[0033] Step 2.4, calculation of the semantic proportion of the directional rectangle: take the four sides of the largest circumscribed rectangle as the length, and the boundary point transformation coefficient Δ 1 With Δ 2 is the width, set the left, right, top and bottom four rectangles, the area is S 1 , S 2 , S 3 , S 4 ; where S 1 =S 2 =(y 2 -y 1 )×Δ 1 , S 3 =S 4 =(x 2 -x 1 )×Δ 2 ; Calculate the number of window pixels in each rectangle and find its ratio to the total number of pixels in the rectangle r 1 、r 2 、r 3 、r 4 ;

[0034] Step 2.5, scale decision and rectangle correction: If r 1 Minimum, then (x 1 +Δ 1 ,y 1 ) and (x 2 ,y 2 ) is the diagonal line to make a new rectangle R ′ ; if r 2 Minimum, then (x 1 ,y 1 ) and (x 2 -Δ 1 ,y 2 ) is a diagonal; if r 3 Minimum, then (x 1 ,y 1 +Δ 2 ) and (x 2 ,y 2 ) is a diagonal; if r 4 Minimum, then (x 1 ,y 1 ) and (x 2 ,y 2 -Δ 2 ) is a diagonal line, and the new rectangle is the semantic information range of the current window.

[0035] Furthermore, in the repeated processing module, the specific method of progressive contour correction is as follows:

[0036] Step 3.1: New rectangle semantic pixel ratio threshold determination: For the new rectangle R obtained in step 2 ′ , calculate the number of pixels N in the window 1 ′ The number of pixels N in the non-window ′ 2 , calculate rat ′ =N ′ 2 / N ′ 1 ; if rat ′ is less than the threshold t, then the new rectangle R ′ is the window range; otherwise proceed to the next step;

[0037] Step 3.2, rectangle contour completion process iteration: transform the new rectangle R ′ Instead of the maximum circumscribed rectangle R in step 2.3, perform the operations of steps 2.3, 2.4, and 2.5 until the ratio of the number of non-window pixels to the number of window pixels in the obtained rectangle is less than the threshold t. In this case, the rectangle is the corresponding range of the window and the iteration terminates.

[0038] Furthermore, in the size unification processing module, the specific method for unifying the sizes of windows in the same row is as follows:

[0039] Step 4.1, calculation of semantic information Z_Score of windows in the same row: For windows in the same row, calculate the mean Where N is the number of windows in the row, i is the window number, and X i is the number of pixels in the window; calculate the standard deviation And use Calculate the Z_Score of each window in this row;

[0040] Step 4.2, rectangle range optimization based on Z_Score: For the semantic information of windows in the same row, if the Z_Score of the window is greater than m or less than -m, where m is a constant, its rectangle range is retained without change; the area average of the remaining windows is calculated again, and a window is found among them so that its semantic information rectangle area is closest to the area average, and the semantic information range of the remaining windows in the same row is constrained to this rectangle;

[0041] Step 4.3, row-by-row semantic label optimization: Repeat steps 4.1 and 4.2 for each row of windows to finally obtain the complete semantic label of the building facade texture image.

[0042] The present invention combines the inherent characteristics of building facade textures, based on progressive contour correction and size unification processing, and proposes a semantic completion method for building facade images to obtain complete semantic information that can be used to synthesize universal textures of building facades. Compared with existing methods, the present invention can combine the structural characteristics of building facades, standardize and complete the existing semantic information of building facades, and obtain high-quality semantic information that can be used to synthesize universal textures of building facades. Therefore, the method has important practical value and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0044] Figure 1 It is a general flow chart for the implementation of the present invention;

[0045] Figure 2 Transform the rectangular boundary points with a certain step size constraint in step 2.3;

[0046] Figure 3 Adjust the bounding rectangle of the direction rectangle constraint in step 2.5;

[0047] Figure 4 To obtain a complete semantic label diagram of the building facade texture image;

[0048] Figure 5 Schematic diagram before and after the window semantic completion;

[0049] Figure 6 Comparison of the number of semantic pixels in the window before and after completion. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] like Figure 1 As shown, the embodiment of the present invention provides a semantic completion method applicable to building facade images, which includes four operations: data preparation, rectangular contour completion processing, progressive contour correction, and unification of window sizes in the same industry. Among them, rectangular contour completion processing, progressive contour correction, and unification of window sizes in the same industry are the main innovations of the present invention. The specific implementation method is as follows:

[0052] Step 1, data preparation: at least including a building facade texture image containing windows and a corresponding semantic label generated by an existing building facade image semantic extraction model, wherein the semantic label at least includes semantic information of the background and the window;

[0053] Step 2: Rectangular outline completion. The specific implementation method is as follows:

[0054] The semantic labels extracted in step 1 may be missing or redundant at the boundary of the window, so we use rectangular contour completion to complete the semantic information of the window.

[0055] Step 2.1, extract the maximum bounding rectangle R for each window in the semantic label.

[0056] Step 2.2, rectangular semantic pixel ratio threshold determination: Calculate the window pixel N in the maximum bounding rectangle R 1 and the number of non-window pixels N 2 , calculate the ratio of non-window pixels to window pixels, rat, where rat = N 2 ÷N 1 ,like Figure 2 In the left image, the black part is the window pixel and the blank part is the non-window pixel. Figure 2 Right picture and Figure 3 The left and right images of only show the borders, without distinguishing the colors. If rat is less than the set threshold t, the rectangle is the window range; otherwise, proceed to the next step.

[0057] Step 2.3, rectangle boundary point transformation calculation: Get the coordinates of the upper left point of the maximum circumscribed rectangle in step 2.2 (x 1 ,y 1 ) and the lower right corner coordinate (x 2 ,y 2 ), set the boundary point transformation coefficient Δ 1 , Δ 2 With the constant factor l, Δ 1 / (x 2 -x 1 )=Δ 2 / (y 2 -y 1 )=l, calculate the coordinates (x 1 +Δ 1 ,y 1 +Δ 2 ) and (x 2 -Δ 1 ,y 2 -Δ 2 ),like Figure 2 As shown in the figure on the right, (Δ 1 / Δ 2 )=(x 2 -x 1 ) / (y 2 -y 1 ).

[0058] Step 2.4: Calculation of the semantic proportion of the directional rectangle: Take the four sides of the largest circumscribed rectangle as the length, and the boundary point transformation coefficient Δ 1 With Δ 2 is the width, set the left, right, top and bottom four rectangles, the area is S 1 , S 2 , S 3 , S 4 ,like Figure 2 As shown in the figure on the right; where S 1 =S 2 =(y 2 -y 1 )×Δ 1 , S 3 =S 4 =(x 2 -x 1 )×Δ 2 . Calculate the number of window pixels in each rectangle and find its ratio to the total number of pixels in the rectangle r 1 、r 2 、r 3 、r 4 .

[0059] Step 2.5, scale decision and rectangle correction: If r 1 Minimum, then (x 1 +Δ 1 ,y 1 ) and (x 2 ,y 2 ) is the diagonal line to make a new rectangle R ′ ; if r 2 Minimum, then (x 1 ,y 1 ) and (x 2 -Δ 1 ,y 2 ) is a diagonal; if r 3 Minimum, then (x 1 ,y 1 +Δ 2 ) and (x 2 ,y 2 ) is a diagonal; if r 4 Minimum, then (x 1 ,y 1 ) and (x 2 ,y 2 -Δ 2 ) is a diagonal line. The new rectangle is the semantic information range of the current window. Figure 3 As shown, at this time r 1 The smallest, so the left rectangle is removed to get the new rectangular range.

[0060] Step 3: The specific implementation method of progressive contour correction is:

[0061] Repeat the operation in step 2, and continuously complete the rectangular outline of the window semantic information according to the ratio of the number of window pixels to the number of non-window pixels in the semantic information, until the semantic information of the window meets the requirements.

[0062] Step 3.1: New rectangle semantic pixel ratio threshold determination: For the new rectangle R obtained in step 2 ′ , calculate the number of pixels N in the window 1 ′ The number of pixels N in the non-window ′ 2 , calculate the ratio of non-window pixels to window pixels rat ′ , where rat ′ =N ′ 2 ÷N ′ 1 ; if rat ′ is less than the threshold t, then the new rectangle R ′ is the window range; otherwise, proceed to the next step.

[0063] Step 3.2, rectangle contour completion process iteration: transform the new rectangle R ′ Instead of the maximum circumscribed rectangle R in step 2.3, perform the operations of steps 2.3, 2.4, and 2.5 until the ratio of the number of non-window pixels to the number of window pixels in the obtained rectangle is less than the threshold t. In this case, the rectangle is the corresponding range of the window and the iteration terminates.

[0064] Step 4: The specific implementation method of unifying the sizes of windows in the same row is:

[0065] For building facade images, the texture sizes of windows in the same row are generally the same. However, in reality, the texture sizes of windows in the same row may be different due to human factors or appearance factors. Therefore, we selectively unify the sizes of windows in the same row.

[0066] Step 4.1, calculation of semantic information Z_Score of windows in the same row: For windows in the same row, calculate the mean Where N is the number of windows in the row, i is the window number, and X i is the number of pixels in the window. Calculate the standard deviation And use Calculate the Z_Score of each window in this row;

[0067] Step 4.2, rectangle range optimization based on Z_Score: For the semantic information of windows in the same row, if the Z_Score of the window is greater than 3 or less than -3, its rectangle range is retained without change; the area average of the remaining windows is calculated again, and a window is found among them whose semantic information rectangle area is closest to the average, and the semantic information range of the remaining windows in the same row is constrained to this rectangle;

[0068] Step 4.3, row-by-row semantic label optimization: Repeat steps 4.1 and 4.2 for each row of windows to finally obtain the complete semantic label of the building facade texture image.

[0069] The final result is Figure 4 As shown in the figure, the left picture is the building facade texture image, and the right picture is the complete semantic information obtained. For multiple different building texture images, we count the number of window pixels in the semantic label of the texture image before completion and the number of window pixels in the semantic label of the texture image after completion. The window semantic labels before and after completion are as follows: Figure 5 As shown in the figure, the black part is the semantic information range of the window, and the pixel number comparison result is as follows Figure 6 shown.

[0070] In specific operations, the method proposed in the present invention can be implemented with the aid of computer software technology to achieve automated process execution. It should be emphasized that these implementations are within the scope of protection of the present invention and can be adopted by professionals in the technical field.

[0071] In another aspect, an embodiment of the present invention further provides a semantic completion system applicable to building facade images, comprising the following modules:

[0072] A data preparation module, used for preparing data, wherein the data at least includes a texture image of a building facade containing windows and a corresponding semantic label, wherein the semantic label at least includes semantic information of a background and a window;

[0073] A rectangular outline completion processing module is used to complete the semantic information of the window by using the rectangular outline completion processing;

[0074] A repeating processing module is used to repeat the operation of the rectangular outline completion processing module, and continuously perform rectangular outline completion processing on the semantic information of the window according to the ratio of the number of window pixels to the number of non-window pixels in the semantic information, until the semantic information of the window meets the requirements;

[0075] The size unification processing module is used to selectively unify the size of windows in the same row, and finally obtain the complete semantic label of the building facade texture image.

[0076] The specific implementation method of each module is the same as each step and will not be described in detail in the present invention.

[0077] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A semantic completion method for building facade images, characterized in that: The following steps are involved: Step 1, data preparation: at least including a building facade texture image containing windows and a corresponding semantic label, the semantic label at least including semantic information of the background and the window; Step 2, using rectangular outline completion processing to complete the semantic information of the window; Step 3, repeating the operation of step 2, and continuously performing rectangular contour completion processing on the semantic information of the window according to the ratio of the number of window pixels to the number of non-window pixels in the semantic information, until the semantic information of the window meets the requirements; Step 4: Selectively resize the windows in the same row to obtain a complete semantic label for the building facade texture image.

2. A semantic completion method for building facade images according to claim 1, characterized in that: The semantic labels in step 1 are obtained through the building facade image semantic extraction model.

3. The semantic completion method for building facade images according to claim 1, characterized in that: In step 2, the specific method of rectangular outline completion is as follows: Step 2.1, extract the maximum bounding rectangle R for each window in the semantic label; Step 2.2, rectangle semantic pixel ratio threshold determination: calculate the number of window pixels N1 and non-window pixels N2 in the maximum bounding rectangle R, and calculate the ratio of non-window pixels to window pixels rat, where rat = N2 / N1; if rat is less than the set threshold t, the rectangle is a window range; otherwise, proceed to the next step; Step 2.3, rectangle boundary point transformation calculation: obtain the coordinates of the upper left point (x1, y1) and the lower right corner (x2, y2) of the maximum circumscribed rectangle in step 2.2, set the boundary point transformation coefficients Δ1, Δ2 and the change constant factor l, so that Δ1 / (x2-x1)=Δ2 / (y2-y1)=l, calculate the coordinates (x1+Δ1, y1+Δ2) and (x2-Δ1, y2-Δ2), where (Δ1 / Δ2)=(x2-x1) / (y2-y1); Step 2.4, calculation of the semantic proportion of directional rectangles: take the four sides of the largest circumscribed rectangle as the length, and the boundary point transformation coefficients Δ1 and Δ2 as the width, set the left, right, top and bottom rectangles with areas S1, S2, S3 and S4; where S1 = S2 = (y2-y1) × Δ1, S3 = S4 = (x2-x1) × Δ2; calculate the number of window pixels in each rectangle, and calculate its ratio to the total number of pixels in the rectangle r1, r2, r3 and r4; Step 2.5, proportion decision and rectangle correction: if r1 is the smallest, then a new rectangle R′ is made with (x1+Δ1, y1) and (X2, y2) as diagonals; if r2 is the smallest, then (x1, y1) and (x2-Δ1, y2) are made as diagonals; if r3 is the smallest, then (x1, y1+Δ2) and (X2, y2) are made as diagonals; if r4 is the smallest, then (x1, y1) and (x2, y2-Δ2) are made as diagonals. The new rectangle is the semantic information range of the current window.

4. The semantic completion method for building facade images according to claim 3, characterized in that: In step 3, the specific method of progressive contour correction is as follows: Step 3.1, threshold determination of semantic pixel ratio of new rectangle: for the new rectangle R′ obtained in step 2, calculate the number of window pixels N′1 and the number of non-window pixels N′2, and calculate rat′=N′2 / N′1; if rat′ is less than the threshold t, the new rectangle R′ is considered to be the window range; otherwise, proceed to the next step; Step 3.2, rectangle contour completion processing iteration: replace the maximum circumscribed rectangle R in step 2.3 with the new rectangle R′, and perform the operations of steps 2.3, 2.4, and 2.5 until the ratio of the number of non-window pixels to the number of window pixels in the obtained rectangle is less than the threshold t, then the rectangle is the corresponding range of the window, and the iteration terminates.

5. The semantic completion method for building facade images according to claim 1, characterized in that: In step 4, the specific method of unifying the sizes of windows in the same row is as follows: Step 4.1, calculation of semantic information Z_Score of windows in the same row: For windows in the same row, calculate the mean Where N is the number of windows in the row, i is the window number, and X i is the number of pixels of the window; Calculate standard deviation And use Calculate the Z_Score of each window in this row; Step 4.2, rectangle range optimization based on Z_Score: For the semantic information of windows in the same row, if the Z_Score of the window is greater than m or less than -m, where m is a constant, its rectangle range is retained without change; the area average of the remaining windows is calculated again, and a window is found among them so that its semantic information rectangle area is closest to the area average, and the semantic information range of the remaining windows in the same row is constrained to this rectangle; Step 4.3, row-by-row semantic label optimization: Repeat steps 4.1 and 4.2 for each row of windows to finally obtain the complete semantic label of the building facade texture image.

6. A semantic completion system for building facade images, characterized in that: Includes the following modules: A data preparation module, used for preparing data, wherein the data at least includes a texture image of a building facade containing windows and a corresponding semantic label, wherein the semantic label at least includes semantic information of a background and a window; A rectangular outline completion processing module is used to complete the semantic information of the window by using the rectangular outline completion processing; A repeating processing module is used to repeat the operation of the rectangular outline completion processing module, and continuously perform rectangular outline completion processing on the semantic information of the window according to the ratio of the number of window pixels to the number of non-window pixels in the semantic information, until the semantic information of the window meets the requirements; The size unification processing module is used to selectively unify the size of windows in the same row, and finally obtain the complete semantic label of the building facade texture image.

7. The semantic completion system for building facade images according to claim 6, characterized in that: The semantic labels in the data preparation module are obtained through the building facade image semantic extraction model.

8. The semantic completion system for building facade images according to claim 6, characterized in that: In the rectangular contour completion processing module, the specific method of rectangular contour completion processing is as follows: Step 2.1, extract the maximum bounding rectangle R for each window in the semantic label; Step 2.2, rectangle semantic pixel ratio threshold determination: calculate the number of window pixels N1 and non-window pixels N2 in the maximum bounding rectangle R, and calculate the ratio of non-window pixels to window pixels rat, where rat = N2 / N1; if rat is less than the set threshold t, the rectangle is a window range; otherwise, proceed to the next step; Step 2.3, rectangle boundary point transformation calculation: obtain the coordinates of the upper left point (x1, y1) and the lower right corner (x2, y2) of the maximum circumscribed rectangle in step 2.2, set the boundary point transformation coefficients Δ1, Δ2 and the change constant factor l, so that Δ1 / (x2-x1)=Δ2 / (y2-y1)=l, calculate the coordinates (x1+Δ1, y1+Δ2) and (x2-Δ1, y2-Δ2), where (Δ1 / Δ2)=(x2-x1) / (y2-y1); Step 2.4, calculation of the semantic proportion of directional rectangles: take the four sides of the largest circumscribed rectangle as the length, and the boundary point transformation coefficients Δ1 and Δ2 as the width, set the left, right, top and bottom rectangles with areas S1, S2, S3 and S4; where S1 = S2 = (y2-y1) × Δ1, S3 = S4 = (x2-x1) × Δ2; calculate the number of window pixels in each rectangle, and calculate its ratio to the total number of pixels in the rectangle r1, r2, r3 and r4; Step 2.5, proportion decision and rectangle correction: if r1 is the smallest, then a new rectangle R′ is made with (x1+Δ1, y1) and (X2, y2) as diagonals; if r2 is the smallest, then (x1, y1) and (x2-Δ1, y2) are made as diagonals; if r3 is the smallest, then (x1, y1+Δ2) and (x2, y2) are made as diagonals; if r4 is the smallest, then (x1, y1) and (x2, y2-Δ2) are made as diagonals. The new rectangle is the semantic information range of the current window.

9. The semantic completion system for building facade images according to claim 6, characterized in that: In the repetitive processing module, the specific method of progressive contour correction is as follows: Step 3.1, threshold determination of semantic pixel ratio of new rectangle: for the new rectangle R′ obtained in step 2, calculate the number of window pixels N′1 and the number of non-window pixels N′2, and calculate rat′=N′2 / N′1; if rat′ is less than the threshold t, the new rectangle R′ is considered to be the window range; otherwise, proceed to the next step; Step 3.2, rectangle contour completion processing iteration: replace the maximum circumscribed rectangle R in step 2.3 with the new rectangle R′, and perform the operations of steps 2.3, 2.4, and 2.5 until the ratio of the number of non-window pixels to the number of window pixels in the obtained rectangle is less than the threshold t, then the rectangle is the corresponding range of the window, and the iteration terminates.

10. The semantic completion system for building facade images according to claim 6, characterized in that: In the size unification processing module, the specific method of unifying the sizes of windows in the same row is as follows: Step 4.1, calculation of semantic information Z_Score of windows in the same row: For windows in the same row, calculate the mean Where N is the number of windows in the row, i is the window number, and X i is the number of pixels of the window; Calculate standard deviation And use Calculate the Z_Score of each window in this row; Step 4.2, rectangle range optimization based on Z_Score: For the semantic information of windows in the same row, if the Z_Score of the window is greater than m or less than -m, where m is a constant, its rectangle range is retained without change; the area average of the remaining windows is calculated again, and a window is found among them so that its semantic information rectangle area is closest to the area average, and the semantic information range of the remaining windows in the same row is constrained to this rectangle; Step 4.3, row-by-row semantic label optimization: Repeat steps 4.1 and 4.2 for each row of windows to finally obtain the complete semantic label of the building facade texture image.

Images