Building recognition model training method and end-to-end building recognition method

Abstract

This disclosure provides a training method for a building recognition model and an end-to-end building recognition method, relating to the field of computer technology, particularly to the fields of mapping, large-scale models, data processing, and artificial intelligence. The specific implementation scheme is as follows: A sample image is input into the building recognition model to obtain the predicted top-face mask of the building instance and the predicted position offset of the bottom face relative to the top face of the building instance; a first edge loss is determined between the predicted top-face mask and the ground truth top-face mask, and an offset loss is determined between the predicted position offset and the ground truth position offset; based on the first edge loss and the offset loss, the model parameters of the building recognition model are optimized.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of mapping, large models, data processing, and artificial intelligence. Background Technology

[0002] With the rapid development of applications such as autonomous driving maps, urban digitization, 3D reconstruction, and commercial site selection analysis, more and more application scenarios are no longer satisfied with obtaining only a rough location of the building's top surface from images (such as satellite images). Instead, they require precise, paired images of the building's top and bottom surfaces. Obtaining paired building top and bottom surfaces is beneficial for more accurately describing the building's footprint, occlusion relationships, and spatial adjacency with roads. These applications place higher demands on the positional accuracy, contour integrity, and mass production capabilities of the bottom surface.

[0003] However, some images, such as satellite images, directly show the top of buildings, while the bottom is often impossible to observe directly due to imaging angle, obstruction, and the complexity of building form. As a result, the bottom outline has long relied on multi-model calculation and rule correction, which is inefficient and difficult to meet production needs. Summary of the Invention

[0004] This disclosure provides a training method for a building recognition model and an end-to-end building recognition method.

[0005] According to one aspect of this disclosure, a method for training a building recognition model is provided, comprising: Input the sample image into the building recognition model to obtain the top mask prediction map of the building instance and the position offset prediction value of the bottom surface of the building instance relative to the top surface. Determine the first edge loss between the top mask prediction map and the top mask ground truth, and determine the offset loss between the position offset prediction value and the position offset ground truth; The model parameters of the building recognition model are optimized based on the first edge loss and offset loss.

[0006] According to another aspect of this disclosure, a training apparatus for a building recognition model is provided, comprising: The first output module is used to input the sample image into the building recognition model to obtain the top mask prediction map of the building instance and the position offset prediction value of the bottom surface of the building instance relative to the top surface. The first loss determination module is used to determine the first edge loss between the top surface mask prediction map and the top surface mask true value, and to determine the offset loss between the position offset prediction value and the position offset true value. The optimization module is used to optimize the model parameters of the building recognition model based on the first edge loss and offset loss.

[0007] According to one aspect of this disclosure, an end-to-end building recognition method is applied to a building recognition model trained using a training method for the building recognition model. The method includes: Input the image to be identified into the building recognition model to obtain the top mask of at least one candidate building in the image to be identified, as well as the offset value of the bottom surface of the building relative to the top surface of the building. The target building is determined from at least one candidate building; Based on the mask image and offset value of the target building, determine the bottom surface of the target building.

[0008] According to another aspect of this disclosure, an end-to-end building recognition device is provided, applied to a building recognition model trained by a building recognition model training method. The device includes: The second output module is used to input the image to be identified into the building recognition model to obtain the top mask image of at least one candidate building in the image to be identified output by the building recognition model, as well as the offset value of the bottom surface of the building relative to the top surface of the building. The first determining module is used to determine the target building from at least one candidate building; The second determining module is used to determine the bottom surface of the target building based on the mask image and offset value of the target building.

[0009] According to another aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and The memory is communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0010] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0011] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a flowchart illustrating a method for training a building recognition model according to an embodiment of the present disclosure; Figure 2 This is a schematic flowchart illustrating the process of determining the first edge loss between the top surface mask prediction map and the top surface mask ground truth according to an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of a first network according to another embodiment of the present disclosure; Figure 4 This is a schematic diagram of the process of obtaining a second edge loss by determining the loss between the edge shape of the target mask image and the true value of the top mask according to an embodiment of the present disclosure; Figure 5 This is a schematic flowchart of an end-to-end building identification method according to an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of a training device for a building recognition model according to an embodiment of the present disclosure; Figure 7 This is a schematic diagram of the structure of an end-to-end building identification device according to an embodiment of the present disclosure; Figure 8 This is a block diagram of an electronic device used to implement the training method for the building recognition model and / or the end-to-end building recognition method of the embodiments of this disclosure. Detailed Implementation

[0014] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] The terms “first,” “second,” etc., used in this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0016] It should be noted that, unless it is explicitly stated that there is a sequential order of execution between different operations, or that there is a sequential order of execution between different operations in terms of technical implementation, the execution order between multiple operations may not be significant, and multiple operations may be executed simultaneously.

[0017] In related technologies, a "top-first, bottom-later, multi-model, multi-stage" pipeline mechanism is commonly used to identify the top and bottom surfaces of a building. Taking satellite imagery as an example, the relevant technical implementation is as follows: First, using a conventional semantic segmentation network (such as UNet (U-Net, a semantic segmentation network) etc.), output the binary representation of the building's top surface on the satellite image; Then, using data such as DSM (Digital Surface Model) / DEM (Digital Elevation Model), stereo pairs, and LiDAR (Light Detection and Ranging), the building height or roof height is estimated. Then, using simple geometric models (such as projecting the roof center along the normal line and estimating the offset based on the building height and solar altitude angle), the top surface is "moved" to the bottom surface position through rules or additional models; Finally, morphological, rule-based filtering, and overlay correction with roads and other parameters are used to try to bring the bottom surface back to a reasonable position.

[0018] In the methods described above, many scenarios still require manual editing and quality control. Furthermore, in this process, "top surface recognition, building height estimation, bottom surface calculation, and vector post-processing" are split into multiple independent modules.

[0019] In view of this, in order to improve the efficiency of building base identification, this embodiment first trains a building identification model, and then uses the model to extract the top and bottom surfaces of buildings in the image. In this embodiment, by performing end-to-end training on the building identification model, the trained building identification model can automatically and stably identify the outline of the building's top surface directly from the image (such as remote sensing imagery), and infer the location and shape of the building's base surface, so as to support large-scale, low-cost, and highly consistent map production and spatial intelligence applications.

[0020] like Figure 1 The diagram shown is a flowchart illustrating the training method for the building recognition model provided in this disclosure, including the following: S101, Input the sample image into the building recognition model to obtain the top mask prediction map of the building instance and the position offset prediction value of the bottom surface of the building instance relative to the top surface.

[0021] The building recognition model in this disclosure is an AI (Artificial Intelligence) model built on a neural network, which supports instance-level segmentation. In this disclosure, a building instance refers to a single building in a sample image. One building instance corresponds to one building.

[0022] The sample images are pre-collected image data containing buildings, which are used as training data to train the building recognition model. Each sample image corresponds to a label file, and the label content includes the ground truth value of the top mask of the building instance and the ground truth value of the position offset of the bottom surface relative to the top surface.

[0023] Both the predicted and true positional offset values ​​describe the offset of the building's base relative to its top surface. The true positional offset value can be expressed using an image coordinate system.

[0024] In practice, sample images can be used as input to the building recognition model. After processing through a series of neural network layers, the model outputs a top mask prediction map of the building instance and a predicted value of the position offset of the bottom surface of the building instance relative to the top surface.

[0025] The top-side mask prediction image is a predicted mask image generated by the building recognition model for the top-side area of ​​a building. It is used to delineate the boundaries between the building's top surface and the background and non-top-side areas in the image, representing the predicted range and outline of the building's top surface in pixel-level regions. The mask image is a binary image, where the value of each pixel indicates whether that location belongs to the top surface of the building, usually represented by 0 and 1.

[0026] The predicted position offset of the bottom surface relative to the top surface is a numerical result calculated by the model based on image features. It represents the positional deviation of the building's bottom surface relative to its corresponding top surface in the image coordinate system. For example, the direction and magnitude of the offset can be represented by a two-dimensional vector.

[0027] S102, determine the first edge loss between the top surface mask prediction map and the top surface mask ground truth, and determine the offset loss between the position offset prediction value and the position offset ground truth.

[0028] Here, the ground truth top mask is the actual mask image of the building instance's top surface, manually labeled during the sample image annotation process. The first edge loss measures the difference between the first edge of the building segmented from the sample image by the top mask prediction and the second edge of the building segmented from the sample image by the ground truth top mask. In other words, the first edge loss constrains edge consistency, which can improve the accuracy of top surface segmentation for buildings with irregular shapes.

[0029] The ground truth positional offset is the actual positional offset of the bottom surface of a building instance relative to its top surface, manually labeled during the annotation process of the sample image. The offset loss is used to measure the difference between the predicted positional offset and the ground truth positional offset.

[0030] S103, based on the first edge loss and offset loss, optimize the model parameters of the building recognition model.

[0031] In implementation, the first edge loss and offset loss can be fused. If there are other losses, multiple losses can also be fused to construct the total loss that optimizes the building recognition model. Then, the model parameters of the building recognition model are optimized through multiple rounds of iteration using the backpropagation algorithm.

[0032] In this embodiment, by inputting sample images into the building recognition model, the top mask prediction map and the position offset prediction value of the bottom surface relative to the top surface of the building instance are output simultaneously. The first edge loss between the top mask prediction map and the ground truth, and the offset loss between the position offset prediction value and the ground truth are calculated respectively. Finally, the model parameters are jointly optimized based on the two types of losses, so that the building recognition model can learn to predict the top outline and bottom spatial position information of the building end-to-end. This can effectively improve the efficiency and accuracy of the building recognition model in outputting the bottom surface of the building, thereby meeting the needs of high-efficiency production.

[0033] In this embodiment of the disclosure, a first edge loss is determined between the top-face mask prediction map and the top-face mask ground truth, such as... Figure 2 As shown, it includes the following: S201, based on the edge detection operator, the top surface mask prediction map is processed to obtain the first gradient map.

[0034] Edge detection operators are algorithms used to detect edge information in images. Examples include the Sobel (Sobel operator) and the Canny (Canny edge detector) operator. Essentially, they work by performing a convolution operation on the image and calculating the gradient in different directions, thereby highlighting the edges of buildings and other structures within the image.

[0035] In implementation, Sobel's filters can be used to convolve the top mask prediction map, resulting in a first gradient map containing image gradient information. The gradient represents the rate of change of pixel values ​​in the image; edges typically correspond to areas with large pixel value changes. Therefore, the first gradient map can highlight the edge positions in the top mask prediction map.

[0036] S202, based on the edge detection operator, the ground truth of the top surface mask is processed to obtain the second gradient map.

[0037] Similarly, a Sobel filter can be used to convolve the ground truth of the top surface mask to obtain a second gradient map containing image gradient information. This second gradient map represents the true edge gradient features of the building's top surface and serves as a benchmark for subsequent error comparison.

[0038] S203, determine the first edge intensity map of the first gradient map, and determine the second edge intensity map of the second gradient map.

[0039] Edge intensity is the gradient value corresponding to each pixel in the gradient map, which represents the strength of the edge at the corresponding pixel. The edge intensity map is obtained by fusing the gradient values ​​in different directions in the gradient map. In practice, the first edge intensity map of the first gradient map is determined by expression (1): (1) In expression (1), Represents the first edge intensity map; This represents the gradient components of the first gradient plot along the x-axis. This represents the gradient component of the first gradient map along the y-axis.

[0040] Similarly, the second edge intensity map of the second gradient map can be determined by expression (2): (2) In expression (2), This represents the second edge intensity map; This represents the gradient components of the second gradient plot along the x-axis. This represents the gradient component of the second gradient map along the y-axis.

[0041] S204, determine the loss between the first edge intensity map and the second edge intensity map to obtain the first edge loss.

[0042] In practice, the first edge loss can be described by expression (3): (3) In expression (3), This represents the first edge loss obtained; Represents the coordinate position of a pixel in the edge intensity map; This represents the effective boundary region obtained by threshold filtering; This represents the edge intensity value of the first edge intensity map at coordinates (x, y); This represents the edge intensity value of the second edge intensity map at coordinates (x, y); This represents the L1 norm.

[0043] In this embodiment, by employing a unified edge detection operator to process the predicted top-face mask image and the ground truth top-face mask to obtain corresponding gradient maps, the edge gradient change features of the predicted mask and the ground truth mask can be captured, avoiding loss measurement distortion caused by differences in detection operators. By extracting an edge intensity map from the gradient map, the abstract gradient change can be transformed into an intuitive quantitative indicator of edge saliency, achieving pixel-level comparison between the predicted edge and the ground truth edge in the same dimension. By calculating the loss between the first edge intensity map and the second edge intensity map, the first edge loss is obtained, which can focus on the edge contour error of the building's top surface, effectively improving the building recognition model's ability to learn the details of the building's top surface edges, thereby improving the accuracy of top and bottom surface prediction.

[0044] In this embodiment of the disclosure, to further improve the accuracy of top surface segmentation, the building recognition model is implemented using a cascaded network. In implementation, the building recognition model includes a first network and a second network. The first network is used to perform instance-level segmentation on the sample image and identify the top surface mask prediction map and position offset prediction value of at least one building instance. For each building instance, the second network is used to extract the top surface mask of the building instance from the segmentation map of the building instance in the sample image, obtaining an optimized top surface mask map of the building instance; the segmentation map is obtained based on the top surface mask prediction map of the building instance.

[0045] Instance-level segmentation refers to the process of distinguishing not only different categories of target regions in an image, but also different individual targets within the same category.

[0046] In implementation, a first network can be used to segment building instances from sample images, and then a second network can be used to refine the segmentation map to obtain higher-quality building top surface information. To ensure that the second network can reuse the feature learning logic of the first network to reduce development and adaptation costs, improve inference efficiency, and avoid overfitting when masking the top surface of building instances, the first and second networks can use the same model architecture (the subsequent prediction heads can be different), and the first network has more parameters than the second network.

[0047] Some possible implementations, such as Figure 3 The diagram shown is a schematic representation of the structure of the first network, including: The backbone network is used to extract multi-scale features from sample images; The pixel network is used to decode the multi-scale features extracted by the backbone network to obtain the first feature map; The decoding network represents potential building instances using a set of queries. It can employ a Mask2Former (Masked-attention Mask Transformer) architecture based on a Transformer decoder, representing potential building instances through a Mask Query mechanism. Here, the Query consists of learnable instance query vectors. The decoding network's input information includes the first feature and the learnable queries, and its output is decoded features. like Figure 3 As shown, multiple heads are provided, each sharing the same input (i.e., decoding features), and each decoding head outputs different results, including: The top surface splitting head (mask head) is used to output a top surface mask for each query (i.e., building instance); Offset Header: Used to output a position offset prediction value for each query, that is, the offset information of the bottom surface relative to the top surface; Furthermore, to improve the accuracy of the top and bottom surface segmentation, in embodiments of this disclosure, such as... Figure 3 As shown, it can also include a Class Header, which outputs a class probability for each query, mainly used to distinguish between foreground (buildings) and background (non-buildings).

[0048] It should be noted that in the cascaded second network, the offset prediction head can be removed, while the top surface segmentation head and classification head are retained.

[0049] Therefore, the first network outputs the top mask prediction map corresponding to each building instance through the top segmentation head, outputs the corresponding category probability through the classification head, and further outputs the two-dimensional position offset prediction value of the corresponding building instance through multilayer MLP (Multilayer Perceptron) fitting based on the offset prediction head, thereby identifying the top mask prediction map and position offset prediction value of at least one building instance.

[0050] In implementation, the first network outputs the predicted top-face mask value for each candidate building, along with its prediction confidence score. Candidate buildings with confidence scores greater than a threshold are selected as building instances for processing by the second network. For each building instance, the top-face mask prediction image output by the first network is first resized and padded to create a segmentation image. Then, the second network processes the segmentation image of each building instance to extract the top-face mask more accurately, resulting in an optimized top-face mask image for the building instance.

[0051] In this embodiment, the building recognition model includes a first network and a second network. It employs a hierarchical processing approach of coarse segmentation and fine optimization. The first network performs instance-level segmentation and outputs a preliminary top-surface mask prediction map and position offset prediction value, enabling rapid localization of building instances and prediction of their basic structures. The second network then performs fine-grained mask extraction and optimization for each building instance, obtaining a more clearly defined optimized top-surface mask map, thereby effectively improving the edge segmentation accuracy of the building recognition model.

[0052] In this embodiment of the disclosure, in order to further optimize the model parameters of the building recognition model and improve the recognition accuracy of the top and bottom surfaces, geometric features in the edges, such as pixels at corners / high curvature / concave / convex locations, are treated as "high-value pixels." This allows for a larger gradient in the segmentation loss at these locations, preventing the model from achieving low loss even by simply smoothing out the edges with a smooth arc.

[0053] Specifically, a second edge loss can be obtained by determining the loss between the edge shapes of the target mask image and the ground truth top mask; the second edge loss is used to optimize the model parameters of the building recognition model; the target mask image includes the top mask prediction image and / or the top mask optimization image.

[0054] In implementation, the first network can be trained first based on the first edge loss and offset loss until the performance of the first network converges, at which point the parameters of the first network are fixed. Then, only the second network is trained, using the second edge loss as the optimization basis, and the model parameters of the second network are adjusted separately through the backpropagation algorithm to avoid disturbing the converged parameters of the first network.

[0055] In some embodiments, after the first network has been initially trained, a second edge loss may be introduced to globally fine-tune the model parameters of the first and second networks.

[0056] In some embodiments, the first and second networks can be jointly trained end-to-end without splitting the training phase. The total model loss is constructed by fusing the second edge loss with various losses described above, such as the first edge loss and offset loss (e.g., weighted summation). Based on this total loss, all model parameters of the first and second networks are simultaneously adjusted using the backpropagation algorithm.

[0057] In this embodiment of the disclosure, a second edge loss is obtained by calculating the difference in edge shape between the target mask image and the true value of the top mask. The parameters of the building recognition model are then optimized using this second edge loss, which can effectively improve the building recognition model's ability to learn and restore the building outline, providing a more reliable foundation for accurately determining the top boundary coordinates and locating the bottom surface of the building.

[0058] In practical implementation, the loss between the edge shapes of the target mask image and the true value of the top mask is determined to obtain the second edge loss, such as... Figure 4 As shown, it includes the following: S401, Determine the bump intensity of the edges of building instances in the sample image.

[0059] Edge convexity strength refers to the numerical index used to represent the degree of protrusion or concavity of the contour edge of the top surface mask of a building instance, relative to the reference edge. A higher value indicates more significant convexity or concavity deformation, while a lower value indicates a smoother edge.

[0060] During implementation, determining the bump intensity of the edges of building instances in the sample image can be achieved through the following steps: Step A1: Solve for the normal direction of the second gradient map of the edge of the true value of the top surface mask to obtain the normal direction map; In practice, the normal direction of the second gradient map at the edge of the true value of the top mask is obtained and can be described by expression (4): (4) In expression (4), Indicates the normal direction angle; Represents the arctangent function; This represents the gradient value of the second gradient plot along the y-axis. This represents the gradient value of the second gradient map in the x-axis direction.

[0061] During implementation, the normal direction is solved by calculating the second gradient map at the edge of the true value of the top surface mask, and the normal direction angle is obtained. Then, the normal direction angles of each pixel are arranged according to the image coordinates to obtain the normal direction map.

[0062] Step A2: Determine the intensity of the change in the normal direction in the normal direction diagram to obtain the concave-convex intensity.

[0063] In practice, this unevenness strength can be described by expression (5): (5) In expression (5), C represents the obtained concavity / convexity strength; The cosine component of the normal direction angle is represented, which maps the angle to a continuous horizontal direction component to avoid periodic jumps in the angle; It represents the sine component of the normal direction angle, that is, the angle is mapped to a continuous vertical direction component to avoid periodic jumps in the angle; This represents the gradient operator, which reflects the local rate of change of the function on the image plane; It indicates the degree of local variation of the cosine component of the normal direction angle in the horizontal direction component; It indicates the degree of local variation of the sinusoidal component of the normal direction angle in the vertical component; This represents the L2 norm.

[0064] In this embodiment, the normal direction map is obtained by solving the normal direction of the second gradient map of the true edge of the top mask, and the concavity and convexity intensity is determined according to the change intensity of the normal direction. This can accurately quantify the key shape features of the building edge, such as convexity, concavity and corner, and provide a reliable basis for subsequent weight allocation to the boundary pixels. This allows the building recognition model to pay more attention to important areas with drastic changes in edge contour during training.

[0065] S402, Extract the boundary band of building instances in the sample image.

[0066] A boundary band refers to a strip-shaped area extending a certain width around the edge contour of a building instance, along both the inner and outer sides of the edge contour. This strip-shaped area covers the edge pixels of the building instance and its neighboring pixels, and is used to highlight or position the transition area between the building and the background.

[0067] In practice, extracting the boundary bands of building instances in the sample image can be achieved based on the following steps: Step B1: Determine the edge intensity of the second gradient map of the edge of the true value of the top surface mask to obtain the second edge intensity map; In practice, the edge strength of the second gradient map of the edge that determines the true value of the top mask can be described by expression (6): (6) In expression (6), The edge strength of the second gradient map representing the edge of the true value of the top face mask; This represents calculating the gradient with respect to the true value GT of the top surface mask; This indicates taking the magnitude of the gradient; This represents the gradient component of the true value of the top mask along the x-axis. This represents the gradient component of the true value of the top mask along the y-axis.

[0068] In practice, after determining the edge intensity of the second gradient map of the edge of the true value of the top mask, the second edge intensity map is obtained by arranging the edge intensity values ​​of all pixels according to the image coordinates into a two-dimensional grayscale map.

[0069] Step B2: Perform a pooling operation on the second edge intensity map to obtain the boundary band of the building instance.

[0070] In practice, max pooling can be used on the second edge intensity map. For example, convolution with a certain radius (such as a 3x3 window) can expand the image of the edge region by a width (such as 5-10 pixels) to form a wider boundary band, which can be used as the boundary band of the resulting building instance.

[0071] In this embodiment, a second edge intensity map is obtained by first determining the edge intensity of the second gradient map of the edge of the true value of the top surface mask, and then a pooling operation is performed on the second edge intensity map to obtain the boundary band of the building instance. This can accurately locate and extract the boundary region of the building top surface, reasonably expand the effective pixel range near the edge to form a stable and continuous boundary band region, and provide a reliable processing range for subsequent calculation of boundary pixel weights and strengthening edge shape constraints. This allows the building recognition model to focus more on the building boundary region during training, thereby improving the accuracy of top surface segmentation and pose offset prediction.

[0072] S403 determines the weights of multiple pixels in the true value of the top mask based on the boundary band and the intensity of the concavity and convexity; the weights are positively correlated with the boundary band and the intensity of the concavity and convexity.

[0073] In implementation, the weights of multiple pixels in the true value of the top mask can be expressed by expression (7): (7) In expression (7), B represents the weights of multiple pixels in the ground truth of the top mask; B represents the extraction of the boundary band of the building instance in the sample image. This indicates the intensity of the concavity and convexity of the edges of building instances in the sample image; This indicates an adjustable hyperparameter used to control the strength of the weighting; This represents the normalized value for the unevenness intensity. Wherein, The weights of the segmentation loss for each pixel are determined. Using W makes the model more sensitive to complex boundaries (such as corners, high curvature, and concavity / convexity).

[0074] The weights are positively correlated with the boundary band and the convexity intensity, meaning that pixels within the boundary band and pixels with high convexity intensity are assigned higher weights. For example, the weights can be calculated based on the distance of a pixel from the edge and the convexity intensity; the closer the pixel is to the edge and the higher the convexity intensity, the higher the weight.

[0075] S404, based on the keypoint weights, the mask loss between the target mask image and the ground truth of the top mask, determines the second edge loss.

[0076] In practice, the second edge loss can be described by expression (8): (8) In expression (8), L represents the determined second edge loss; This indicates the determination of the weights of multiple pixels in the true value of the top mask; Represents the target mask image; Indicates the true value of the top face mask; This represents the mask loss between the target mask and the true value of the top mask.

[0077] In this embodiment, by first determining the edge convexity intensity of building instances in the sample image and extracting boundary bands, and then assigning corresponding weights to each pixel in the ground truth of the top surface mask based on the boundary bands and convexity intensity, higher attention can be paid to the interest points or key points of the building edge lines, strengthening the constraint on the details of the contour geometric features. Furthermore, based on these weights and the mask loss between the target mask image and the ground truth, a second edge loss is determined. This allows the building recognition model to focus more on the edge shape and contour structure of the building's top surface during training, making the final output top surface mask image more closely match the real building boundary and improving the accuracy of top and bottom surface segmentation of building instances.

[0078] In this embodiment of the disclosure, to further optimize the model parameters of the building recognition model, at least one of the following losses may be determined: (1) Mask segmentation loss between the target mask image and the true value of the top mask; In implementation, for each building instance, the Dice coefficient of a single instance is determined, and then the overall Dice Loss is determined based on the Dice coefficient of the single instance. Based on this, the mask segmentation loss can be described by expression (9): (9) In expression (9), represents the Dice coefficient of the i-th building instance in the sample image, used to measure the overlap between the target mask (predicted mask) and the top face mask (ground mask (GT)) on the i-th building instance. The closer the Dice coefficient is to 1, the more accurate the prediction; k represents the pixel index; This represents the target mask value (i.e., the value in the target mask image) of the k-th pixel of the i-th building instance. Represents the true value of the top face mask of the k-th pixel of the i-th building instance; This represents the intersection of the target mask value and the true value of the top mask; This represents the sum of squares of all pixels in the target mask; This represents the sum of squares of all pixels in the top face mask; This represents a smoothing term, used to avoid a denominator of 0; The mask segmentation loss represents the difference between the target mask image and the ground truth top mask; N represents the total number of building instances.

[0079] (2) The classification loss between the candidate instances obtained by instance-level segmentation of the sample image and the classification loss between the ground truth and the classification loss; In practice, the classification loss can be described by expression (10): (10) In expression (10), Indicates the classification loss; N represents the total number of building instances; K represents the total number of classification categories; i represents the instance index; k represents the category index; This represents the classification prediction value of the i-th instance in the k-th class; This represents the truth value of the classification of the i-th instance in the k-th class; It represents the natural logarithm.

[0080] It should be noted that the mask segmentation loss and classification loss are used to optimize the model parameters of the building recognition model.

[0081] In this embodiment of the disclosure, by determining at least one of the mask segmentation loss and classification loss and using it to optimize the model parameters, the building recognition model can be constrained from the dimensions of mask region accuracy and instance classification reliability, respectively. This enables the building recognition model to more accurately segment the top surface region of the building and correctly distinguish between building and non-building instances, thereby improving the completeness and accuracy of the output top surface mask image.

[0082] In this embodiment of the disclosure, L1 loss or L2 loss can be used to determine the loss between the predicted position offset and the true position offset.

[0083] In other embodiments, to improve the accuracy of position offset prediction, determining the offset loss between the predicted position offset value and the true position offset value can be achieved based on the following steps: Step C1: Determine the distance between the pixel-level predicted position offset and the true position offset to obtain the pixel-level offset distance; Step C2: For each pixel, if the pixel's offset distance is within the threshold range, the sub-loss of the pixel adopts L2 loss; Step C3: If the offset distance is outside the threshold range, the sub-loss of the pixel adopts L1 loss; Step C4: Determine the offset loss based on the sub-loss of each pixel.

[0084] In practice, the offset loss is determined based on the sub-loss of each pixel, and can be described by expression (11): (11) In expression (11), Indicates offset loss; Represents the SmoothL1 (Huber) loss function; This represents the predicted position offset. represents the true position offset; x represents the offset error of a single element between the predicted and true position offset values. Indicates the range of offset distance thresholds; Indicates L2 loss; This represents L1 loss.

[0085] In practice, positional offsets can be described in a structured manner, such as using two elements: x-direction offset and y-direction offset. Alternatively, more complex structures can be used, such as assigning a positional offset in both the x and y directions to each pixel. Therefore, the number of elements in the positional offset depends on how it is represented.

[0086] In this embodiment of the disclosure, selecting an appropriate loss at a suitable time can improve the accuracy of position offset prediction while also taking training efficiency into account. For example, appropriately using L1 loss can suppress outliers and avoid gradient explosion, while appropriately introducing L2 loss can refine regression and make training more stable.

[0087] Based on the same technical concept, this disclosure also provides an end-to-end building recognition method, applied to a building recognition model trained using the above method, such as... Figure 5 As shown, it includes the following: S501, input the image to be identified into the building recognition model to obtain the top mask image of at least one candidate building in the image to be identified output by the building recognition model, and the offset value of the bottom surface of the building relative to the top surface of the building.

[0088] The image to be identified refers to the original image without annotations that needs to be used for building identification. For example, satellite images, aerial images, remote sensing images, and other image data containing buildings.

[0089] In practice, the image to be identified can be input into the building recognition model trained by the method described above. The model will rely on the instance segmentation and prediction capabilities of the first network and the optimization capabilities of the second network to output the top mask image of at least one candidate building in the image to be identified, as well as the offset value of the bottom surface of the building relative to the top surface of the building.

[0090] S502, determine the target building from at least one candidate building.

[0091] That is, the target building can be identified from multiple candidate buildings detected in the image to be identified by the building recognition model.

[0092] S503, Based on the mask image and offset value of the target building, determine the bottom surface of the target building.

[0093] During implementation, the key coordinates and outline range of the top surface can be determined based on the top surface mask of the target building. Then, by combining the corresponding offset value of the bottom surface relative to the top surface, the outline coordinates and position information of the bottom surface of the building can be derived through coordinate offset calculation. Finally, the synchronous positioning of the top and bottom surfaces of the building can be achieved, and the height of the building can also be expressed.

[0094] In this embodiment, by inputting the image to be recognized into a trained building recognition model, the model directly outputs the top mask image of the candidate building and the offset value of the bottom surface relative to the top surface. This enables rapid end-to-end building recognition, obtaining a clear outline of the top surface region that fits the real edge, as well as its spatial position relationship. By filtering and determining the target building from the candidate buildings, false detections, duplicates, or scattered recognition results can be eliminated, ensuring that the final object used for processing is a valid and regular building instance. Furthermore, based on the optimized top mask image and offset value of the target building, the accuracy of determining the corresponding building bottom surface can be improved, better restoring the spatial structure of the building's top and bottom surfaces.

[0095] In this embodiment of the disclosure, determining the target building from at least one candidate building can be achieved based on the following steps: Step D1: Based on the top mask image of each candidate building, segment each candidate building from the image to be identified; During implementation, the top surface mask image of each candidate building can be extracted first through edge detection or contour tracking algorithms to obtain an ordered sequence of boundary points.

[0096] The ordered sequence of boundary points is then simplified using the Douglas-Peucker algorithm or a similar method to preserve the main inflection points of the contour. A distance threshold can be set during boundary point simplification to control the degree of simplification.

[0097] Next, the simplified angle distribution of each side is calculated, the dominant direction of the building is identified (usually 0° and 90°), and the sides close to the dominant direction are forcibly aligned to preset angles (such as 0°, 45°, and 90°) to eliminate the small deflection caused by noise.

[0098] Finally, the pixel coordinates are converted into image coordinates and organized in the format POLYGON((x1y1,x2y2,...,xnyn)) to ensure that the first and last points coincide, forming a closed polygon, which is used as the ordered sequence of boundary points.

[0099] Step D2: If the distance between at least two candidate buildings is less than a first threshold and the difference between the offset values ​​of at least two candidate buildings is less than a second threshold, then merge at least two candidate buildings into the target building.

[0100] In this embodiment of the disclosure, the distance between at least two candidate buildings can refer to the pixel distance between the boundaries of the top mask maps of the two candidate buildings.

[0101] In implementation, if the distance between at least two candidate buildings is less than a first threshold, it indicates that the two buildings are spatially adjacent and may belong to the same building. For example, the top surface mask images of the two output candidate buildings are adjacent, and the boundary pixel distance is less than 5 pixels.

[0102] Similarly, if the difference between the offset values ​​of at least two candidate buildings is less than a preset second threshold, it indicates that the positional changes of the bottom surface relative to the top surface of these candidate buildings are similar, and they may belong to the same building.

[0103] In practice, if both of the above conditions are met, at least two candidate buildings that are spatially close and have similar changes in their base position will be merged into a single target building.

[0104] In this embodiment, by segmenting the corresponding candidate buildings from the image to be identified based on the top-face mask image of each candidate building, scattered building areas that may be identified separately can be located individually, facilitating subsequent accurate judgment and processing. By merging multiple candidate buildings with a spatial distance less than a first threshold and an offset difference less than a second threshold, multiple fragmented areas that originally belonged to the same building but were predicted separately by the model can be reintegrated into a complete target building, effectively solving the problem of the same building being identified multiple times, thereby improving the completeness and accuracy of building identification.

[0105] Furthermore, to avoid unreasonable merging and ensure that the final identified target buildings more accurately reflect the actual building situation, the determination of the target buildings can be further optimized based on the following conditions.

[0106] During implementation, buildings can be merged step by step. For example, one candidate building is processed at a time, determining whether it should be merged with the building to be merged. If the area obtained after merging is greater than a preset area, the merging operation for that step is canceled, and the building to be merged and the candidate building before the last merging operation are respectively determined as the target buildings. That is, when merging one candidate building at a time to obtain the target building, if the area difference between the target building and the reference building is greater than a preset area, at least two candidate buildings are respectively determined as the target buildings; the reference building is any one of the two buildings before the current merging step.

[0107] Specifically, if the area difference between the target building and the reference building is greater than the preset area, it indicates a significant difference in area between the merged target building and the reference building. This difference may suggest that these candidate buildings should not actually be merged into a single entity, but rather are independent and distinct buildings. Therefore, at least two candidate buildings that were originally intended to be merged can be designated as target buildings. For example, building 1 and building 2 are merged to obtain building Z1. If the area difference between Z1 and building 1 is large, the merge is cancelled; if the area difference between Z1 and building 2 is large, the merge is cancelled. If the area differences between Z1 and both building 1 and building 2 are small, the merge takes effect. If there is another building 3 that is close to Z1 and has a similar positional offset, then building 3 is merged into Z1 to obtain Z2. The area differences between Z2 and Z1, and between Z2 and building 3 are compared. If either area difference is large, the merge is cancelled, and the final target buildings are Z1 and building 3.

[0108] In this embodiment of the disclosure, when the area difference between the target building and the reference building exceeds a preset threshold, multiple candidate buildings are identified as the target building respectively. This is used to distinguish different building instances with significant area differences, avoid mismerging or misjudgment, and improve the completeness and accuracy of identification.

[0109] In other embodiments, there may be voids that cause a large area difference. Therefore, if the area difference between the target building and the reference building is greater than the preset area, the middle hole of the target building can also be identified. The reference building is any one of at least two buildings.

[0110] In other words, if the area difference between the target building and the reference building is greater than the preset area, it indicates a significant difference in area between the two. This difference may indicate that the target building has a more complex structure and may contain special areas, such as central openings. Therefore, in this case, it is necessary to further identify the central openings of the target building.

[0111] A central opening refers to a hollow area existing within the top surface of a target building. For example, a building with a skylight will have a hollow area in its top surface.

[0112] Therefore, if the area difference between the target building and the reference building is greater than the preset area, it indicates a significant difference in area between the two. The identification of internal openings within the building may suggest that these candidate buildings should not actually be merged into a single entity, but rather are likely independent buildings. Therefore, at least two candidate buildings that were originally intended to be merged can be designated as the target building.

[0113] In this embodiment of the disclosure, by identifying the intermediate hole when the area difference between the target building and the reference building exceeds a preset area, it is possible to accurately distinguish the internal hollow structure of adjacent independent buildings from that of the same building, avoid erroneous merging or segmentation, and make the top mask image more closely match the shape and internal structure of the real building, thereby improving the completeness and accuracy of top and bottom surface identification.

[0114] In this embodiment of the disclosure, the top surface boundary coordinates of the target building's top surface in the image to be identified are determined based on the top surface mask image of the target building.

[0115] During implementation, after the building recognition model outputs the top surface mask of the target building in one go, the top surface mask can be used to identify and determine the top surface boundary coordinates of the target building in the image to be recognized through contour extraction, coordinate mapping and other methods.

[0116] In this embodiment of the disclosure, the top surface boundary coordinates of the target building in the image to be identified are determined based on the top surface mask image of the target building, which can provide a reliable numerical basis for the subsequent calculation of the bottom surface coordinates of the building.

[0117] That is, in this embodiment of the disclosure, the spatial relationship between the top and bottom surfaces is explicitly modeled: instead of relying on later geometric rules or external models to calculate the bottom surface, a simple and unified vector structure is used to describe the projection offset of the building under the imaging geometry, thereby achieving a direct conversion from the top surface to the bottom surface.

[0118] During implementation, the data representation format for each building is as follows: POLYGON((x1y1,x2y2,...,xnyn)) OFFSET((dx,dy)); Wherein, POLYGON represents the outline of the building's top surface, obtained based on the top surface mask; OFFSET((dx,dy)) represents the displacement of the building's overall bottom surface relative to its top surface; it can be extended to a polygonal expression with offsets per vertex / per pixel to support more complex structures.

[0119] In summary, the solution provided by the embodiments of this disclosure has the following advantages: 1. Building expression level: The spatial relationship between the top and bottom surfaces is explicitly depicted using a single vector (Offset).

[0120] 2. From the perspective of data annotation, a joint annotation format of "top polygon + offset" is proposed, which makes the spatial relationship between the top and bottom surfaces clearly depicted in the annotation.

[0121] 3. From the model structure level, within the instance segmentation framework, an instance-level offset prediction head (or offset vector field) is introduced, which enables the simultaneous output of the top and bottom surfaces of a building in a single inference.

[0122] 4. From the training mechanism level, by using top surface segmentation loss + offset regression loss + top / bottom surface consistency constraint, the model can truly learn "spatial geometric mapping from top surface to bottom surface", rather than simple rule extrapolation.

[0123] 5. From an engineering application perspective, it significantly simplifies the production line, reduces reliance on multi-module and multi-source data, lowers manual editing costs, and improves the accuracy and stability of building base positioning and shape representation.

[0124] Therefore, compared with the "top-first, bottom-later, multi-module extrapolation" approach in related technologies, the embodiments disclosed in this paper have advantages such as improved prediction accuracy of the top and bottom surfaces, more controllable errors, simpler system, and stronger engineering feasibility. They are applicable to various scenarios such as vehicle maps, commercial maps, and urban simulations, and have high engineering value.

[0125] Based on the same technical concept, this disclosure also provides a training device 600 for a building recognition model, such as... Figure 6 As shown, it includes: The first output module 601 is used to input the sample image into the building recognition model to obtain the top mask prediction map of the building instance and the position offset prediction value of the bottom surface of the building instance relative to the top surface. The first loss determination module 602 is used to determine the first edge loss between the top surface mask prediction map and the top surface mask true value, and to determine the offset loss between the position offset prediction value and the position offset true value. The optimization module 603 is used to optimize the model parameters of the building recognition model based on the first edge loss and offset loss.

[0126] In some embodiments, the first loss determination module includes: The first processing unit is used to process the top surface mask prediction map based on the edge detection operator to obtain the first gradient map; The second processing unit is used to process the true value of the top surface mask based on the edge detection operator to obtain the second gradient map; The first determining unit is used to determine a first edge intensity map of the first gradient map and a second edge intensity map of the second gradient map. The second determining unit is used to determine the loss between the first edge intensity map and the second edge intensity map to obtain the first edge loss.

[0127] In some embodiments, the building recognition model includes a first network and a second network, wherein the first network is used to perform instance-level segmentation on sample images and identify the top mask prediction map and position offset prediction value of at least one building instance; For each building instance, the second network is used to extract the top mask of the building instance from the segmentation map of the building instance in the sample image to obtain the optimized top mask map of the building instance; the segmentation map is obtained based on the predicted top mask map of the building instance.

[0128] In some embodiments, a second loss determination module is further included, for: The loss between the edge shapes of the target mask image and the ground truth top mask is determined to obtain the second edge loss; the second edge loss is used to optimize the model parameters of the building recognition model; the target mask image includes the top mask prediction image and / or the top mask optimization image.

[0129] In some embodiments, the second loss determination module includes: The third determining unit is used to determine the concavity and convexity intensity of the edges of the building instance in the sample image; Extraction unit, used to extract the boundary band of building instances in the sample image; The fourth determining unit is used to determine the weights of multiple pixels in the true value of the top surface mask based on the boundary band and the concavity and convexity intensity; the weights are positively correlated with the boundary band and the concavity and convexity intensity. The fifth determining unit is used to determine the second edge loss based on the key point weights, the mask loss between the target mask image and the top mask ground truth.

[0130] In some embodiments, the third determining unit is specifically used for: The normal direction is obtained by solving the second gradient map of the edge of the true value of the top mask; The intensity of the change in the normal direction in the normal direction diagram is determined to obtain the concavity / convexity intensity.

[0131] In some embodiments, the extraction unit is specifically used for: The edge intensity of the second gradient map of the edge of the true value of the top surface mask is determined to obtain the second edge intensity map; The boundary band of the building instance is obtained by performing a pooling operation on the second edge intensity map.

[0132] In some embodiments, a third loss module is also included, for: Determine at least one of the following losses: The mask segmentation loss between the target mask image and the ground truth top mask; the target mask image includes the top mask prediction image and / or the top mask optimization image; The classification loss between the candidate instances obtained by instance-level segmentation of the sample image and the classification loss between the ground truth classification value; Masking segmentation loss and classification loss are used to optimize the model parameters of the building recognition model.

[0133] In some embodiments, the first loss determination module further includes: The sixth determining unit is used to determine the distance between the pixel-level predicted position offset value and the true position offset value, so as to obtain the pixel-level offset distance; The first loss unit is used for each pixel, where the pixel's offset distance is within a threshold range, and the sub-loss of the pixel adopts L2 loss. The second loss unit is used when the offset distance is outside the threshold range, and the sub-loss of the pixel adopts L1 loss. The seventh determining unit is used to determine the offset loss based on the sub-loss of each pixel.

[0134] Based on the same technical concept, this disclosure also provides an end-to-end building recognition device 700, applied to a building recognition model trained by a building recognition model training method, such as... Figure 7 As shown, it includes: The second output module 701 is used to input the image to be identified into the building recognition model to obtain the top mask image of at least one candidate building in the image to be identified output by the building recognition model, as well as the offset value of the bottom surface of the building relative to the top surface of the building. The first determining module 702 is used to determine the target building from at least one candidate building; The second determining module 703 is used to determine the bottom surface of the target building based on the mask image and offset value of the target building.

[0135] In some embodiments, the first determining module includes: The recognition unit is used to segment each candidate building from the image to be recognized based on the top mask image of each candidate building; A merging unit is used to merge at least two candidate buildings into a target building when the distance between at least two candidate buildings is less than a first threshold and the difference between the offset values ​​of at least two candidate buildings is less than a second threshold.

[0136] In some embodiments, a first optimization module is further included, for: In each step of merging a candidate building to obtain the target building, if the area difference between the target building and the reference building is greater than the preset area, at least two candidate buildings will be determined as the target building respectively; the reference building is any one of the two buildings before the current step of merging.

[0137] In some embodiments, a second optimization module is further included, for: If the area difference between the target building and the reference building is greater than the preset area, identify the central hole in the target building; the reference building is any one of at least two buildings.

[0138] In some embodiments, a top surface recognition module is further included, for: Identify the top surface boundary coordinates of the target building's top surface in the image to be identified based on the top surface mask map of the target building.

[0139] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0140] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0142] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0143] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0144] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0145] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as methods for training building recognition models and / or end-to-end building recognition methods. For example, in some embodiments, the methods for training building recognition models and / or end-to-end building recognition methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the methods for training building recognition models and / or end-to-end building recognition methods described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a training method for the building recognition model and / or an end-to-end building recognition method.

[0146] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0150] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0151] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0152] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a building recognition model, comprising: Input the sample image into the building recognition model to obtain the top mask prediction map of the building instance and the position offset prediction value of the bottom surface of the building instance relative to the top surface, which are output by the building recognition model. Determine the first edge loss between the top surface mask prediction map and the top surface mask ground truth, and determine the offset loss between the position offset prediction value and the position offset ground truth; Based on the first edge loss and the offset loss, the model parameters of the building recognition model are optimized.

2. The method according to claim 1, wherein, The determination of the first edge loss between the top-face mask prediction map and the top-face mask ground truth includes: The top surface mask prediction map is processed based on the edge detection operator to obtain the first gradient map; The second gradient map is obtained by processing the top surface mask ground truth value based on the edge detection operator; Determine a first edge intensity map of the first gradient map, and determine a second edge intensity map of the second gradient map; The loss between the first edge intensity map and the second edge intensity map is determined to obtain the first edge loss.

3. The method according to claim 1, wherein the building recognition model comprises a first network and a second network, the first network being used to perform instance-level segmentation on the sample image and identify the top surface mask prediction map and the position offset prediction value of at least one building instance; For each building instance, the second network is used to extract the top mask of the building instance from the segmentation map of the building instance in the sample image to obtain an optimized top mask map of the building instance; the segmentation map is obtained based on the predicted top mask map of the building instance.

4. The method according to claim 3, further comprising: The loss between the edge shape of the target mask image and the ground truth of the top mask is determined to obtain a second edge loss; the second edge loss is used to optimize the model parameters of the building recognition model; the target mask image includes the predicted top mask image and / or the optimized top mask image.

5. The method according to claim 4, wherein, The loss between determining the edge shape of the target mask image and the true value of the top surface mask, to obtain the second edge loss, includes: Determine the convexity / concavity intensity of the edges of the building instance in the sample image; Extract the boundary band of the building instance in the sample image; Based on the boundary band and the convexity / concave intensity, the weights of multiple pixels in the true value of the top surface mask are determined; the weights are positively correlated with the boundary band and the convexity / concave intensity. Based on the keypoint weights and the mask loss between the target mask image and the top surface mask ground truth, the second edge loss is determined.

6. The method according to claim 5, wherein, Determining the convexity / concavity intensity of the edges of the building instance in the sample image includes: The normal direction is obtained by solving the second gradient map of the edge of the true value of the top surface mask; The intensity of the change in the normal direction in the normal direction diagram is determined to obtain the concavity / convexity intensity.

7. The method according to claim 5, wherein, The step of extracting the boundary band of the building instance in the sample image includes: The edge intensity of the second gradient map of the edge of the true value of the top surface mask is determined to obtain the second edge intensity map; The boundary band of the building instance is obtained by performing a pooling operation on the second edge intensity map.

8. The method according to any one of claims 1-7, further comprising: Determine at least one of the following losses: The mask segmentation loss between the target mask image and the true value of the top surface mask; The target mask image includes the top-face mask prediction image and / or the top-face mask optimization image; The classification loss between the candidate instances obtained by instance-level segmentation of the sample image and the classification loss between the ground truth classification value; The mask segmentation loss and the classification loss are used to optimize the model parameters of the building recognition model.

9. The method according to claim 1, wherein, The step of determining the offset loss between the predicted position offset and the true position offset includes: Determine the distance between the pixel-level predicted position offset and the true position offset to obtain the pixel-level offset distance; For each pixel, if the offset distance of the pixel is within the threshold range, the sub-loss of the pixel adopts L2 loss; If the offset distance is outside the threshold range, the sub-loss of the pixel adopts L1 loss; The offset loss is determined based on the sub-loss of each pixel.

10. An end-to-end building recognition method, applied to a building recognition model trained by the method according to any one of claims 1-9, the method comprising: The image to be identified is input into the building recognition model to obtain the top mask image of at least one candidate building in the image to be identified, and the offset value of the bottom surface of the building relative to the top surface of the building, as output by the building recognition model. The target building is determined from the at least one candidate building; Based on the mask image and offset value of the target building, the bottom surface of the target building is determined.

11. The method according to claim 10, wherein, Determining the target building from the at least one candidate building includes: Based on the top mask image of each candidate building, each candidate building is segmented from the image to be identified; If the distance between at least two candidate buildings is less than a first threshold and the difference between the offset values ​​of the at least two candidate buildings is less than a second threshold, the at least two candidate buildings are merged into the target building.

12. The method of claim 11, further comprising: In each step of merging a candidate building to obtain a target building, if the area difference between the target building and the reference building is greater than a preset area, the at least two candidate buildings are respectively determined as the target buildings; The reference building is either of the two buildings that were merged in the current step.

13. The method of claim 11, further comprising: If the area difference between the target building and the reference building is greater than a preset area, identify the central hole in the target building; The reference building is any one of the at least two buildings.

14. The method according to claims 10-13, further comprising: Identify the top surface boundary coordinates of the target building in the image to be identified based on the top surface mask map of the target building.

15. A training device for a building recognition model, comprising: The first output module is used to input the sample image into the building recognition model to obtain the top mask prediction map of the building instance and the position offset prediction value of the bottom surface of the building instance relative to the top surface. The first loss determination module is used to determine the first edge loss between the top surface mask prediction map and the top surface mask true value, and to determine the offset loss between the position offset prediction value and the position offset true value. An optimization module is used to optimize the model parameters of the building recognition model based on the first edge loss and the offset loss.

16. An end-to-end building recognition device, applied to a building recognition model trained by the method described in any one of claims 1-9, the device comprising: The second output module is used to input the image to be identified into the building recognition model to obtain the top mask image of at least one candidate building in the image to be identified, and the offset value of the bottom surface of the building relative to the top surface of the building. A first determining module is used to determine the target building from the at least one candidate building; The second determining module is used to determine the bottom surface of the target building based on the mask image and offset value of the target building.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-14.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-14.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-14.

Images