Building edge regularization method for collaborative optimization of GAN network and image processing

Abstract

A building edge regularization method for collaborative optimization of a GAN network and image processing belongs to the technical field of digital image processing, and is characterized by comprising the following steps: constructing the GAN network; performing regularization algorithm training on the GAN network to determine model parameters; predicting and outputting a regularized building mask by adopting the model parameters; performing mask preprocessing to obtain a denoised binary image; counting connected domains; all connected domains are traversed in sequence for filtering processing; performing edge contour thinning to obtain a connected domain after edge contour thinning; and carrying out edge right angle processing to obtain an optimized building mask. The problems of noise, holes, edge burrs and the like in an original building mask can be effectively solved; the optimization method is stable and reliable, contour redundant points of the optimized building mask are greatly simplified, the right angle features are obvious, the building mask is more in line with the actual building contour, the quality of the building mask directly predicted by a deep learning method is greatly improved, and application of a building intelligent extraction algorithm is accelerated.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image processing, and in particular relates to a building edge regularization method for collaborative optimization of a GAN network and image processing. Background Art

[0002] Traditional intelligent building extraction algorithms based on deep learning methods have no building geometric rule constraints in the prediction stage, resulting in irregular edges of predicted building masks. Therefore, technical problems often occur, such as "other non-building targets are mistakenly identified as building targets, i.e., pseudo targets; irregular edges of building masks, including rounded corners and non-straight edges; the presence of a large number of noise points and holes; redundant pixel points on the mask edges, resulting in complex and inefficient subsequent application calculations" and other technical problems. Summary of the Invention

[0003] The present invention aims to solve the above problems and provide a general building edge regularization method that is collaboratively optimized by GAN network and image processing.

[0004] In a first aspect, the present invention provides a building edge regularization method for collaborative optimization of a GAN network and image processing, comprising: A GAN network consisting of a generator network and a discriminator network is constructed; the generator network is used to generate a regularized mask of the building mask; the discriminator network is used to evaluate the similarity between the regularized mask and the manually annotated ideal mask; the GAN network is trained with a regularization algorithm to determine model parameters; an RGB original image and a grayscale building mask are received, and the regularized building mask is predicted and output using the model parameters; Read the regularized building mask and perform mask preprocessing to obtain a binary image after noise reduction; Set a threshold for the area of ​​connected domains and a threshold for the number of points on edge contours; count the connected domains on the aforementioned binary image; a connected domain is an area in the image consisting of adjacent foreground pixels with the same pixel value; traverse all connected domains in sequence and filter each connected domain; then perform edge contour thinning to obtain a connected domain after edge contour thinning; The connected domain after edge contour thinning is subjected to edge square processing to obtain the optimized building mask.

[0005] Furthermore, the building edge regularization method for collaborative optimization of the GAN network and image processing of the present invention, wherein the sequential traversal of all connected domains and filtering of each connected domain comprises the following steps: Step 21: If the current connected domain has a parent connected domain, filter out the current contour and jump to traverse the next connected domain; otherwise, execute the next step. Step 22: If the area of ​​the current connected domain is less than or equal to the area threshold, filter out the current contour and jump to traverse the next connected domain; otherwise, execute the next step; Step 23: If the number of points on the edge contour of the current connected domain is less than or equal to the threshold, the current contour is filtered out and the process of traversing the next connected domain is skipped; otherwise, the process proceeds to the next step. Step 24: Thinning the edge contours of the connected domain after filtering.

[0006] Furthermore, the building edge regularization method of the GAN network and image processing collaborative optimization of the present invention includes the following steps of performing edge square processing on the connected domain after the edge contour is thinned: Step 31: Coordinate space conversion: convert the contour in the image coordinate system into the contour in the Gaussian plane rectangular coordinate system; Step 32: Set a side length threshold, traverse two adjacent points on the contour under the rectangular coordinates of the Gaussian plane in sequence, and perform the following steps 33 to 35 for each two adjacent points; Step 33: Calculate the length of the side formed by each two adjacent points; Step 34: If the length of two adjacent points is less than the edge length threshold, filter out the neighboring points corresponding to the current point, continue to select the next point as the neighboring point, and jump to step 33 and repeat; otherwise, execute step 35; Step 35: Calculate the azimuth of the edge formed by two adjacent points, expressed in degrees; Step 36: Determine the main direction; determine the direction of the longest side as the main direction, and the azimuth of the main direction as the reference value; Step 37: Direction correction: rotate the sides except the longest side so that they are perpendicular or parallel to the main direction, with the midpoint of the line segment formed by two adjacent points as the center of rotation; Step 38: Adjacent edge correction: Based on the edge set after direction correction in step 37, traverse the two adjacent edges in turn and perform adjacent edge correction; Step 39: Coordinate space conversion; convert the building roof outline obtained based on the Gaussian plane rectangular coordinate system into the building outline in the image coordinate system.

[0007] Furthermore, in the building edge regularization method for collaborative optimization of the GAN network and image processing of the present invention, the direction correction includes the following steps: Step 41: Determine the rotation target; first, calculate the difference between the azimuth of the current side and the reference value; if the absolute value of the difference is less than 45 degrees, rotate the side to be parallel to the longest side; if the absolute value of the difference is greater than or equal to 45 degrees, rotate the side to be perpendicular to the longest side; Step 42: Determine the rotation method; then rotate the first and last points of the side with the midpoint of the side as the center; if the difference in step 41 is greater than 0, rotate the difference clockwise to make the side parallel to the longest side; if the difference in step 41 is less than 0, rotate the difference counterclockwise to make the side perpendicular to the longest side.

[0008] Furthermore, in the building edge regularization method for collaborative optimization of the GAN network and image processing of the present invention, the adjacent edge correction includes the following steps: Step 51: Setting the adjacent edge parallel distance threshold; Step 52: Determine the positional relationship of the adjacent edges. If the positional relationship of the two adjacent edges relative to the main direction is inconsistent, they must be perpendicular, and the intersection of the adjacent edges is calculated. If the positional relationship of the two adjacent edges relative to the main direction is consistent, they must be parallel. Correction of parallel adjacent edges includes the following two methods: If the distance between the two parallel adjacent edges is less than the distance threshold in step 51, the start point and the end point of the next edge are adjusted successively to be in a straight line with the current edge; If the distance between the two parallel adjacent edges is greater than the distance threshold in step 51 , a vertical line is added to be perpendicular to the two adjacent edges.

[0009] Furthermore, in the building edge regularization method of the collaborative optimization of the GAN network and image processing described in the present invention, the regularized building mask is first read during the mask preprocessing; then the noise points are filtered using the median filtering method; and then dilation and erosion processing are used in sequence.

[0010] In a second aspect, the present invention provides a building edge regularization system for collaborative optimization of a GAN network and image processing, comprising a GAN network, a mask preprocessing module, a connected domain analysis module, and an edge squared module; The GAN network consists of a generator network and a discriminator network; the generator network is used to generate a regularized mask of the building mask; the discriminator network is used to evaluate the similarity between the regularized mask and the manually labeled ideal mask; the GAN network is trained with a regularization algorithm to determine model parameters; the GAN network receives an RGB original image and a grayscale building mask, and uses the model parameters to predict and output the regularized building mask; The mask preprocessing module is used to read the regularized building mask output by the GAN network to perform mask preprocessing to obtain a binary image after noise reduction; The connected domain analysis module is used to set a connected domain area threshold and a threshold for the number of points on the edge contour; count the connected domains on the aforementioned binary image; traverse all connected domains in sequence and filter each connected domain; and then obtain a connected domain after edge contour thinning by edge contour thinning; The edge square module is used to perform edge square processing on the connected domain after the edge contour is thinned out to obtain an optimized building mask.

[0011] Furthermore, the GAN network and image processing collaboratively optimized building edge regularization system of the present invention comprises a generator network consisting of three sub-networks: a generator encoder, a reconstruction encoder, and a decoder; the network structure of the generator encoder and the reconstruction encoder consists of a stack of multiple CBL (Conv2d+Norm2d+RELU) and MaxPool2d layers, and the structures are identical except for the number of input channels of the first Conv2d layer; the first Conv2d layer of the generator encoder receives an RGB original image and a grayscale building mask, and the first Conv2d layer of the reconstruction encoder receives a manually labeled grayscale building mask; the decoder consists of a stack of multiple ResidualBlocks (Conv2d+Norm2d+ReLU+Conv2d+Norm2d+ReLU), Upsample, CBL, and the last two layers of Conv2d and Sigmoid; The discriminator network is composed of multiple Conv2d, ReLU, MaxPool2d, Norm2d and finally Sigmoid stacks.

[0012] In a third aspect, the present invention provides a building edge regularization device for collaborative optimization of a GAN network and image processing, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to implement the building edge regularization method for collaborative optimization of a GAN network and image processing as described in the first aspect when executing the computer program.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the building edge regularization method for collaborative optimization of the GAN network and image processing described in the first aspect is implemented.

[0014] The building edge regularization method for collaborative optimization of the GAN network and image processing described in the present invention utilizes the GAN network and multiple image processing methods for collaborative optimization, which can effectively solve the problems of noise, voids, edge burrs, etc. in the original building mask. The optimization method is stable and reliable, and the optimized building mask greatly simplifies the redundant points of the contour, has obvious right-angle features, and is more consistent with the actual building contour. The quality of the building mask directly predicted by the deep learning method is greatly improved, which accelerates the application of the intelligent building extraction algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of the GAN network structure according to an embodiment of the present invention; Figure 2 This is an example diagram of the GAN regularization effect described in an embodiment of the present invention; Figure 3 This is an example diagram of edge contour thinning according to an embodiment of the present invention; Figure 4 This is an example graph of neighbor point traversal according to an embodiment of the present invention; Figure 5 This is an example diagram of direction correction according to an embodiment of the present invention; Figure 6 This is a schematic diagram of parallel adjacent edge adjustment situation 1 according to an embodiment of the present invention; Figure 7 This is a schematic diagram of parallel adjacent edge adjustment situation 2 according to an embodiment of the present invention; Figure 8 This is an example graph of adjacent edge traversal according to an embodiment of the present invention; Figure 9 This is an example diagram of edge squareness according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following is a detailed description of the building edge regularization method for collaborative optimization of the GAN network and image processing described in the present invention through the accompanying drawings and examples.

[0017] Example 1 This embodiment discloses a building edge regularization method for collaborative optimization of a GAN network and image processing, including the following steps: Step S1: GAN network regularization; specifically including: Step S1.1: Construct a GAN network; the constructed GAN network structure is as follows Figure 1 As shown in the figure, it consists of a generator network G and a discriminator network D. The generator network consists of three sub-networks: a generator encoder EG, a reconstruction encoder ER, and a decoder F. The network structure of the generator encoder EG and the reconstruction encoder ER is composed of multiple CBL (Conv2d+Norm2d+RELU) and MaxPool2d layer stacks. Except for the number of input channels of the first layer Conv2d, the other structures are exactly the same. The first layer Conv2d of the generator encoder EG receives the RGB type original image and the grayscale type building mask. The first layer Conv2d of the reconstruction encoder ER receives the manually labeled grayscale type building mask.

[0018] The decoder F consists of multiple ResidualBlocks (Conv2d+Norm2d+ReLU+Conv2d+Norm2d+ReLU), Upsample, CBL, and the last two layers of Conv2d and Sigmoid stacking. The discriminator network D consists of multiple Conv2d, ReLU, MaxPool2d, Norm2d, and finally Sigmoid stacking.

[0019] Step S1.2: GAN regularization algorithm training; During the GAN regularization algorithm training phase, the generator network is used to generate a regularized mask for the building mask. It receives an RGB original image Z, a grayscale mask X, and a manually labeled ideal mask Y. Through the training and learning process, the generator network establishes a mapping relationship from mask X to ideal mask Y. The discriminator network is used to evaluate the similarity between the regularized mask generated by the generator and the ideal building mask. If the similarity is infinite, the generated regularized mask is considered valid, that is, the result is true; otherwise, the generated regularized mask is considered invalid, that is, the result is false.

[0020] Step S1.3: GAN predicts the regularized mask; After the network training in step 12, the network parameters of the encoder EG, reconstruction encoder ER, decoder F, and discriminator D in the generator network G are learned and determined. The prediction phase directly uses the weight parameters of the two sub-networks of encoder EG and decoder F for prediction. It receives the RGB type original image and the grayscale type building mask, and outputs the regularized building mask after prediction. The effect of GAN regularization is shown in the following example. Figure 2 As shown, Figure 2-1 Building masks directly predicted by deep learning algorithms, Figure 2-2 This is the building mask after GAN regularization. Its original rounded corner features are basically optimized to right-angle features, and the mask outline is also changed from a curve to a straight line.

[0021] Step S2: mask pre-processing; specifically including: Step S2.1: Read the regularized mask predicted by GAN, such as Figure 3 As shown, black is the background, white is the foreground, and the foreground is the building roof mask; Step S2.2: Filter out noise points using the median filter method; Step S2.3: Dilation and erosion are used successively to make the mask edge of the building roof smoother, and a binary image after noise reduction is obtained.

[0022] Step S3: Connected domain analysis; specifically including: Step S3.1: Set a threshold for the area of ​​connected domains and a threshold for the number of points on edge contours; and count all connected domains on the binary image. Step S3.2: Traverse all connected components in sequence and perform filtering processing on each connected component; Step S3.3: If the current connected component has a parent connected component, filter out the current contour and jump to Step S3.2 to start traversing the next connected component; otherwise, execute the next Step S3.4; Step S3.4: If the area of the current connected component is less than or equal to the area threshold, filter out the current contour and jump to S3.2 to start traversing the next connected component; otherwise, execute the next Step S3.5; Step S3.5: If the number of points on the edge contour of the current connected component is less than or equal to the number threshold, filter out the current contour and jump to S3.2 to start traversing the next connected component; otherwise, execute the next Step S3.6; Step S3.6: Thin out the edge contours of the filtered connected components.

[0023] In the embodiment of the present disclosure, the Douglas - Peucker algorithm is used to simplify the number of points on the contour to ensure the computational efficiency of building extraction from remote sensing images in business analysis and processing. At the same time, it is required that the polygon formed by the remaining key points approximates the original edge contour shape infinitely.

[0024] The process of implementing edge contour thinning by the Douglas - Peucker algorithm includes: 1) Set the distance threshold T; 2) Connect the starting point Ps and the ending point Pe of the contour to fabricate a straight line Lse; 3) Traverse all points on the contour and calculate the distance Di from point i to Lse in sequence; 4) Find the maximum distance value Dmax and compare Dmax with the distance threshold T; 5) If Dmax < T: Then discard all points except Ps and Pe; use Lse as the approximation of this segment of the edge contour curve, and this segment of the curve is processed; 6) If Dmax ≥ T: Then retain the point corresponding to Dmax, and divide the curve into two parts with this point as the boundary. Repeat using this method for these two parts, that is, repeat steps 1) - 3) until all Dmax < T, and the thinning of this segment of the curve is completed.

[0025] In the embodiment of the present disclosure, during the process of edge contour thinning in Step S3.6, the detailed calculation steps for the perpendicular distance from point x0 to the straight line formed by x1 and x2 are as follows: Step S3.6.1: Construct vectors; respectively construct vector x2 - x1 and vector x1 - x0. Vector x2 - x1 represents the edge pointing from x1 to x2; vector x1 - x0 represents the edge pointing from x0 to x1.

[0026] Step S3.6.2: Calculate the determinant; form the two vectors constructed in step S3.5.1 into a 2×2 matrix and calculate its determinant.

[0027] Step S3.6.3: Calculate the norm; the absolute value of the determinant corresponds to the area of ​​the parallelogram, and the norm of the vector x2-x1 represents the length of the base.

[0028] Step S3.6.4: Find the distance; the area of ​​a parallelogram can also be calculated by multiplying the length of the base by the vertical height. Therefore, dividing the area by the length of the base gives the vertical distance from point x0 to the line formed by x1 and x2.

[0029] The formula for calculating the distance Di from point x0 to the imaginary line formed by points x1 and x2 in steps S3.6.1 to S3.6.4 above can be expressed as follows: The edge contour thinning effect of step S3.6 in this embodiment is as follows: Figure 4 As shown, Figure 4-1 The white outline in the middle is the mask outline of the original building roof predicted by the algorithm, and the small red dots are the key points on the outline retained after simplification by the Douglas-Peucker algorithm. Figure 4-2 The red outline in the middle is the new outline composed of key points. In this example, the original outline is composed of 234 points, and the simplified new outline is composed of only 13 points. However, it can be seen that the polygon formed by them is infinitely close to the original outline.

[0030] Step S4: Edge squareness; Analyze and process the new contour formed in step S3.6 of the connected domain analysis. Specifically, the following steps are performed: Step S4.1: Coordinate space conversion: converting the contour in the image coordinate system into the contour in the Gaussian plane rectangular coordinate system; Step S4.2: Set the edge length threshold, traverse two adjacent points on the contour under the rectangular coordinates of the Gaussian plane in sequence, and perform the following steps S4.3 to S4.5 on each adjacent two points. Figure 5 As shown, the red line segment is the longest side, and the two endpoints of the blue line segment are the neighboring points of each traversal.

[0031] Step S4.3: Calculate the length of each side formed by two adjacent points. For example, the length from point (x1, y1) to point (x2, y2) is calculated using the Euclidean formula: Step S4.4: If the length of two adjacent points is less than the edge length threshold, filter out the neighboring points corresponding to the current point, select the next point as the neighboring point, and jump to step S4.3 and repeat. Otherwise, execute step S4.5; Step S4.5: Calculate the azimuth of the edge formed by two adjacent points, expressed as an angle; Step S4.6: Determine the main direction. Determine the direction of the longest side as the main direction, and the azimuth of the main direction as the reference value; Step S4.7: Direction correction. Rotate all sides except the longest side so that they are perpendicular or parallel to the main direction, using the midpoint of the line segment formed by two adjacent points as the center. The detailed steps are as follows: Step S4.7.1: Determine the rotation target; first calculate the difference between the azimuth of the current side and the reference value; if the absolute value of the difference is less than 45 degrees, rotate the side to be parallel to the longest side; if the absolute value of the difference is greater than or equal to 45 degrees, rotate the side to be perpendicular to the longest side.

[0032] Step S4.7.2: Determine the rotation method; then rotate the first and last points of the side with the midpoint of the side as the center; if the difference in step S4.7.1 is greater than 0, rotate the difference clockwise to make the side parallel to the longest side; if the difference in step S4.7.1 is less than 0, rotate the difference counterclockwise to make the side perpendicular to the longest side.

[0033] In the embodiment of the present disclosure, an example of direction correction in step S4.7 is as follows: Figure 6 As shown in the figure, the white outline is the outline after thinning, the blue marked edge is the edge after direction correction, and the red marked edge is the longest edge.

[0034] Step S4.8: Adjacent edge correction: After the direction correction in step S4.7, some adjacent edges are parallel but not intersecting, or intersecting perpendicularly with multiple right angles. Adjacent edge correction is based on the set of edges after the direction correction in step S4.7, and traverses two adjacent edges in sequence, performing the following analysis and processing; Step S4.8.1: Setting the adjacent edge parallel distance threshold; Step S4.8.2: Determine the positional relationship of adjacent edges. If the positional relationship of the two adjacent edges relative to the principal direction is inconsistent, they must be perpendicular. Calculate the intersection of the adjacent edges. If the positional relationship of the two adjacent edges relative to the principal direction is consistent, they must be parallel. Correction of parallel adjacent edges is handled in the following two cases: Case 1: If the distance between two parallel adjacent edges is less than the distance threshold in step S4.8.1, the next edge (such as Figure 6 The starting and ending points of line segment B are adjusted successively to be in a straight line with the current edge, that is, the two projection points of the starting and ending points of B on line A are found. In this way, after the correction, the four points (the starting and ending points of A, the starting and ending points of B) are in a straight line.

[0035] Case 2: If the distance between two parallel adjacent edges is greater than the distance threshold in step S4.8.1, add a vertical line perpendicular to the two adjacent edges, such as Figure 7 As shown, first calculate the midpoint between the end point of A and the starting point of B. Then add a line segment (orange segment) perpendicular to A and B through the midpoint.

[0036] In the embodiment of the present disclosure, the traversal process of the adjacent edges in step S4.8 is as follows: Figure 8 As shown, Figure 8-1 The edge marked in blue is the edge after the direction is corrected in step S4.7, and the edge marked in red is the longest edge. Figure 8-2 Start to Figure 8-1 4 A total of 13 traversals are completed to traverse all adjacent edges. Figure 9 As shown, the white outline is the outline after thinning, and the green outline is the outline after adjacent edge correction.

[0037] Step S4.9: Coordinate space conversion: Convert the building roof outline obtained in the Gaussian plane rectangular coordinate system into the building outline in the image coordinate system.

[0038] Example 2 The present invention discloses a building edge regularization system for collaborative optimization of a GAN network and image processing, comprising a GAN network, a mask preprocessing module, a connected domain analysis module, and an edge squared module; The GAN network consists of a generator network and a discriminator network; the generator network is used to generate a regularized mask of the building mask; the discriminator network is used to evaluate the similarity between the regularized mask and the manually labeled ideal mask; the GAN network is trained with a regularization algorithm to determine model parameters; the GAN network receives an RGB original image and a grayscale building mask, and uses the model parameters to predict and output the regularized building mask; The mask preprocessing module is used to read the regularized building mask output by the GAN network to perform mask preprocessing to obtain a binary image after noise reduction; The connected domain analysis module is used to set a connected domain area threshold and a threshold for the number of points on the edge contour; count the connected domains on the aforementioned binary image; traverse all connected domains in sequence and filter each connected domain; and then obtain a connected domain after edge contour thinning by edge contour thinning; The edge square module is used to perform edge square processing on the connected domain after the edge contour is thinned out to obtain an optimized building mask.

[0039] In the embodiment of the present disclosure, Figure 1As shown in the figure, the generator network consists of three sub-networks: a generation encoder, a reconstruction encoder, and a decoder. The network structure of the generator encoder and the reconstruction encoder consists of multiple CBL (Conv2d+Norm2d+RELU) and MaxPool2d layers stacked together. Except for the number of input channels of the first layer Conv2d, the other structures are exactly the same. The first layer Conv2d of the generation encoder receives the RGB type original image and the grayscale type building mask, and the first layer Conv2d of the reconstruction encoder receives the manually labeled grayscale type building mask.

[0040] The decoder consists of multiple ResidualBlocks (Conv2d+Norm2d+ReLU+Conv2d+Norm2d+ReLU), Upsample, CBL, and the last two layers of Conv2d and Sigmoid stacking; the discriminator network consists of multiple Conv2d, ReLU, MaxPool2d, Norm2d, and the final Sigmoid stacking.

[0041] The process of implementing the building edge regularization system for collaborative optimization of the GAN network and image processing described in this embodiment is the same as the building edge regularization method for collaborative optimization of the GAN network and image processing described in the aforementioned embodiment 1, and will not be repeated here.

[0042] Example 3 This embodiment discloses a building edge regularization device for collaborative optimization of a GAN network and image processing, comprising a memory and a processor; the memory is configured to store a computer program; and the processor is configured to, when executing the computer program, implement the building edge regularization method for collaborative optimization of a GAN network and image processing as described in the first embodiment. The specific steps of the edge regularization method are the same as those in the first embodiment and are not further described here.

[0043] Example 4 This embodiment discloses a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the building edge regularization method for collaborative optimization of a GAN network and image processing as described in the first embodiment. The specific steps of the edge regularization method are the same as those in the first embodiment and are not further described here.

[0044] The computer described in the embodiment of the present application can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can read or a data storage device such as a server, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (DVD)) or a semiconductor medium (for example, a solid-state drive (SSD)), etc. The software formed by the computer storage code can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or the like.

[0045] The functional modules in the various embodiments of the present application can be integrated into a processing unit or module, or each module can exist physically separately, or two or more modules can be integrated into a unit or module. In the above embodiments, they can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part.

[0046] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A building edge regularization method for collaborative optimization of GAN network and image processing, characterized by include: Construct a GAN network consisting of a generator network and a discriminator network; The generator network is used to generate a regularized mask of the building mask; The discriminator network is used to evaluate the similarity between the regularized mask and the manually annotated ideal mask; The GAN network is trained with a regularization algorithm to determine model parameters. An RGB original image and a grayscale building mask are received, and the model parameters are used to predict and output a regularized building mask. Read the regularized building mask and perform mask preprocessing to obtain a binary image after noise reduction; Set the threshold of the area of ​​the connected domain and the number of points on the edge contour; Counting the connected domains on the aforementioned binary image; traversing all connected domains in turn, filtering each connected domain; and then obtaining the connected domain after edge contour thinning by edge contour thinning; The connected domain after edge contour thinning is subjected to edge square processing to obtain the optimized building mask.

2. The building edge regularization method for collaborative optimization of GAN network and image processing according to claim 1, characterized in that: The method of sequentially traversing all connected domains and filtering each connected domain comprises the following steps: Step 21: If the current connected domain has a parent connected domain, filter out the current contour and jump to traverse the next connected domain; otherwise, execute the next step. Step 22: If the area of ​​the current connected domain is less than or equal to the area threshold, filter out the current contour and jump to traverse the next connected domain; otherwise, execute the next step; Step 23: If the number of points on the edge contour of the current connected domain is less than or equal to the threshold, the current contour is filtered out and the process of traversing the next connected domain is skipped; otherwise, the process proceeds to the next step. Step 24: Thinning the edge contours of the connected domain after filtering.

3. The building edge regularization method for collaborative optimization of GAN network and image processing according to claim 1 or 2, characterized in that: The edge square processing of the connected domain after the edge contour is thinned includes the following steps: Step 31: Coordinate space conversion: convert the contour in the image coordinate system into the contour in the Gaussian plane rectangular coordinate system; Step 32: Set a side length threshold, traverse two adjacent points on the contour under the rectangular coordinates of the Gaussian plane in sequence, and perform the following steps 33 to 35 for each two adjacent points; Step 33: Calculate the length of the side formed by each two adjacent points; Step 34: If the length of two adjacent points is less than the edge length threshold, filter out the neighboring points corresponding to the current point, continue to select the next point as the neighboring point, and jump to step 33 and repeat; otherwise, execute step 35; Step 35: Calculate the azimuth of the edge formed by two adjacent points, expressed in degrees; Step 36: Determine the main direction; determine the direction of the longest side as the main direction, and the azimuth of the main direction as the reference value; Step 37: Direction correction: rotate the sides except the longest side so that they are perpendicular or parallel to the main direction, with the midpoint of the line segment formed by two adjacent points as the center of rotation; Step 38: Adjacent edge correction: Based on the edge set after direction correction in step 37, traverse the two adjacent edges in turn and perform adjacent edge correction; Step 39: Coordinate space conversion; convert the building roof outline obtained based on the Gaussian plane rectangular coordinate system into the building outline in the image coordinate system.

4. The building edge regularization method for collaborative optimization of GAN network and image processing according to claim 3 is characterized in that: The direction correction includes the following steps: Step 41: Determine the rotation target; first, calculate the difference between the azimuth of the current side and the reference value; if the absolute value of the difference is less than 45 degrees, rotate the side to be parallel to the longest side; if the absolute value of the difference is greater than or equal to 45 degrees, rotate the side to be perpendicular to the longest side; Step 42: Determine the rotation method; then rotate the first and last points of the side with the midpoint of the side as the center; if the difference in step 41 is greater than 0, rotate the difference clockwise to make the side parallel to the longest side; if the difference in step 41 is less than 0, rotate the difference counterclockwise to make the side perpendicular to the longest side.

5. The building edge regularization method for collaborative optimization of GAN network and image processing according to claim 4 is characterized in that: The adjacent edge correction comprises the following steps: Step 51: Setting the adjacent edge parallel distance threshold; Step 52: Determine the positional relationship of the adjacent edges. If the positional relationship of the two adjacent edges relative to the main direction is inconsistent, they must be perpendicular, and the intersection of the adjacent edges is calculated. If the positional relationship of the two adjacent edges relative to the main direction is consistent, they must be parallel. Correction of parallel adjacent edges includes the following two methods: If the distance between the two parallel adjacent edges is less than the distance threshold in step 51, the start point and the end point of the next edge are adjusted successively to be in a straight line with the current edge; If the distance between the two parallel adjacent edges is greater than the distance threshold in step 51 , a vertical line is added to be perpendicular to the two adjacent edges.

6. The building edge regularization method for collaborative optimization of a GAN network and image processing according to claim 1, characterized in that: During the mask preprocessing, the regularized building mask is first read; then the noise points are filtered using a median filter method; and then dilation and erosion processing are performed in sequence.

7. A building edge regularization system for collaborative optimization of a GAN network and image processing, characterized by: Includes GAN network, mask preprocessing module, connected domain analysis module and edge squared module; The GAN network consists of a generator network and a discriminator network; the generator network is used to generate a regularized mask of the building mask; the discriminator network is used to evaluate the similarity between the regularized mask and the manually annotated ideal mask; The GAN network is trained with a regularization algorithm to determine model parameters. An RGB original image and a grayscale building mask are received, and the model parameters are used to predict and output a regularized building mask. The mask preprocessing module is used to read the regularized building mask output by the GAN network to perform mask preprocessing to obtain a binary image after noise reduction; The connected domain analysis module is used to set a connected domain area threshold and a threshold for the number of points on the edge contour; count the connected domains on the aforementioned binary image; traverse all connected domains in sequence and filter each connected domain; and then obtain a connected domain after edge contour thinning by edge contour thinning; The edge square module is used to perform edge square processing on the connected domain after the edge contour is thinned out to obtain an optimized building mask.

8. The building edge regularization system for collaborative optimization of a GAN network and image processing according to claim 7, characterized in that: The generator network consists of three sub-networks: a generator encoder, a reconstruction encoder, and a decoder. The network structure of the generator encoder and the reconstruction encoder consists of multiple CBL and MaxPool2d layers stacked together. The first layer of the generator encoder receives an RGB original image and a grayscale building mask. The first layer of the reconstruction encoder receives a manually annotated grayscale building mask. The decoder is composed of multiple ResidualBlock, Upsample, CBL and the last two layers of Conv2d and Sigmoid stacking; The discriminator network is composed of multiple Conv2d, ReLU, MaxPool2d, Norm2d and finally Sigmoid stacks.

9. A building edge regularization device for collaborative optimization of GAN network and image processing, characterized in that: The invention comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the building edge regularization method for collaborative optimization of the GAN network and image processing according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the building edge regularization method for collaborative optimization of the GAN network and image processing according to any one of claims 1 to 6 is implemented.

Images