A building three-dimensional reconstruction method and system based on 3D Gaussian sputtering

Abstract

The present application relates to the technical field of 3D Gaussian sputtering reconstruction and geometric accuracy optimization, in particular to a building three-dimensional reconstruction method and system based on 3D Gaussian sputtering, which identifies building right-angle components through a double-channel network, outputs a semantic segmentation graph with geometric mutation characteristics, generates a non-uniform Gaussian kernel distribution, contains a right-angle inflection point ellipsoidal type forbidden area and a wall surface normal direction gradient attenuation kernel density, reconstructs the isotropic kernel into an ellipsoidal type anisotropic kernel aligned with the normal, establishes an independent texture channel, segments the cross-surface Gaussian kernel and clears the texture weight in the overlapping area, introduces a right-angle coplanarity loss function to correct the kernel coplanarity deviation, implants an auxiliary kernel in the edge low-confidence high-deviation area, and optimizes the coordinates through back projection calibration. The system includes building component semantic analysis, adaptive kernel distribution control, parallel texture processing and geometric accuracy enhancement unit, finally outputs a high-precision reconstruction model with right-angle non-distortion and high edge resolution, and adapts to the building construction drawing restoration requirement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of 3D Gaussian sputtering reconstruction and geometric precision optimization, in particular to a building three-dimensional reconstruction method and system based on 3D Gaussian sputtering. BACKGROUND

[0002] 3D Gaussian sputtering reconstruction and geometric precision optimization technology is an important technology, which is specifically applied to building right-angle component three-dimensional reconstruction and building construction drawing restoration links, and the core is to realize high-precision reconstruction by accurately adapting the geometric characteristics of right-angle components and texture mapping rules, to meet the stringent requirements of urban building surveying and mapping on right-angle deviation and edge resolution.

[0003] The current building three-dimensional reconstruction method based on 3D Gaussian sputtering is difficult to adapt to the special structure of right-angle components in the urban building surveying and mapping scene, and cannot meet the precision standard required by the construction drawing restoration. The existing 3D Gaussian sputtering reconstruction method does not design a Gaussian kernel distribution strategy for the geometric mutation characteristics of building right-angle components, but still uses a global uniform distribution and a fixed isotropic Gaussian kernel, resulting in a high Gaussian kernel overlap rate at the right-angle corner point. The originally sharp right angle is excessively smoothed to form a rounded corner, which deviates significantly from the actual right-angle structure. At the same time, the texture mapping consistency of the Gaussian kernels on both sides of the right angle is not verified, and the cross-face mixing of the Gaussian kernels causes the texture of the two sides of the wall to cross-face aliasing, and the reconstructed texture is misaligned with the actual wall texture. The coupling of these two problems causes the reconstruction error to exceed the specified value, which cannot meet the core needs of right-angle deviation and edge detail resolution, and cannot adapt to the application scene of building construction drawing restoration. In order to solve this technical problem, we provide a building three-dimensional reconstruction method and system based on 3D Gaussian sputtering. SUMMARY

[0004] The present application aims to provide a building three-dimensional reconstruction method and system based on 3D Gaussian sputtering to solve the problems raised in the background.

[0005] To achieve the above-mentioned purpose, one of the purposes of the present application is to provide a building three-dimensional reconstruction method based on 3D Gaussian sputtering, comprising the following steps:

[0006] S1, identifying the building right-angle component region in the input image through a deep learning model, including the accurate contour boundary of the wall corner and the door and window frame, and outputting a semantic segmentation map with geometric mutation feature markers;

[0007] S2, for the right angle component region marked in the semantic segmentation map, a non-uniform Gaussian kernel distribution strategy is generated, including prohibiting Gaussian kernel distribution within the radius range of the right angle corner center, and implementing gradient attenuation kernel density strategy in the area extending along the normal direction on both sides of the corner, and replacing the isotropic Gaussian kernel with a direction-constrained Gaussian kernel, that is, the kernel principal axis direction is forced to align with the wall normal;

[0008] S3, independent texture mapping channels are established for the walls on both sides of the right angle component, and a texture boundary checking module is added in GPU parallel calculation, when the Gaussian kernel coverage area crosses the walls on both sides, the Gaussian kernel is automatically divided into two independent sub-kernels and is respectively bound to the corresponding wall texture, and a texture weight zeroing mechanism is implemented for the overlapping area of the sub-kernels;

[0009] S4, a right angle coplanarity loss function based on projection geometry is introduced, the coplanar deviation of the Gaussian kernel center points on both sides of the right angle is detected in real time, and when the included angle between the normal vectors of adjacent Gaussian kernels deviates from the preset angle range, iterative kernel position correction is triggered until the right angle constraint is met;

[0010] S5, an auxiliary Gaussian kernel is implanted within a fixed range of the component edge, the spatial coordinates of the auxiliary kernel are optimized through edge pixel back projection, and finally a reconstructed model meeting the right angle deviation and edge resolution is output.

[0011] The second object of the application is to provide a system for implementing a building three-dimensional reconstruction method based on 3D Gaussian sputtering, comprising:

[0012] The building construction semantic analysis unit integrates a double-channel edge gradient collaborative perception network and a geometric mutation feature fusioner, inputs an original image and outputs a semantic segmentation map carrying a geometric mutation feature tensor, and the tensor includes an error ellipsoid parameter, a wall normal vector threshold and an edge thickness probability distribution histogram;

[0013] The adaptive kernel distribution control unit receives the geometric mutation feature marking of the semantic segmentation map, establishes an ellipsoid space expansion forbidden area at the right angle corner and calculates the adaptive coefficient of the forbidden radius, synchronously generates a kernel density decay coordinate system to drive an exponential density decay curve, and replaces the isotropic Gaussian kernel with an ellipsoid anisotropic kernel with a long-axis radius compression and a short-axis radius expansion according to the edge heat map confidence;

[0014] The parallel texture processing unit deploys a texture boundary checking module in the GPU calculation pipeline, when an ellipsoid anisotropic kernel is detected to cross the semantic segmentation boundary, a kernel voxel is divided to generate a sub-kernel bound to an independent texture channel, and a texture weight zeroing mechanism is implemented based on the comparison of the spatial overlap rate and the kernel density threshold;

[0015] The geometric precision enhancement unit is built-in with a projection geometry coplanarity checker, which calculates an ellipsoid type anisotropic kernel major axis vector cross product module in real time, triggers iterative kernel position migration with the projection origin being the center of the forbidden area, and starts an auxiliary kernel implantation program according to the edge thermal map confidence and coplanar deviation risk, and through edge pixel back projection closed loop calibration, the auxiliary kernel space coordinates are optimized until the color gradient residual error converges.

[0016] Compared with the prior art, the beneficial effects of the present application are:

[0017] The present application generates a non-uniform Gaussian kernel distribution for the right-angle component geometric mutation characteristics, divides the cross-surface Gaussian kernel by combining the texture boundary checking module, and implements texture weight zeroization in the sub-kernel overlap area, assisted by the right-angle coplanarity loss function to correct the kernel coplanar deviation and implantation auxiliary Gaussian kernel to optimize the edge details, so as to achieve the effects of no right-angle and corner distortion, no cross-surface aliasing of wall texture on both sides, and reconstruction error controlled within the accuracy range and meeting the building construction drawing restoration requirements, effectively solving the problem that the existing 3D Gaussian sputtering reconstruction method does not adapt to the right-angle geometric mutation characteristics and does not check the texture mapping consistency, resulting in coupling of right-angle and corner distortion and texture cross-surface aliasing, causing the reconstruction error to exceed the standard. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The overall workflow diagram of the present application is shown in the figure;

[0019] Figure 2 The overall structure schematic diagram of the present application is shown in the figure;

[0020] The meanings of the various labels in the figure are as follows:

[0021] 1, building construction semantic analysis unit; 2, adaptive kernel distribution control unit; 3, parallel texture processing unit; 4, geometric precision enhancement unit. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0023] Please refer to Figure 1 The present embodiment aims to provide a 3D Gaussian sputtering based building three-dimensional reconstruction method, which comprises the following steps:

[0024] S1, identifying the building right-angle component region in the input image by a deep learning model, including the accurate contour boundary of the wall corner and the door and window frame, and outputting a semantic segmentation map with geometric mutation feature markers;

[0025] S2, for the right angle component region marked in the semantic segmentation map, a non-uniform Gaussian kernel distribution strategy is generated, including prohibiting Gaussian kernel distribution within a radius range of the right angle corner center, and implementing a gradient attenuation kernel density strategy within a region extending along the normal direction of the corner on both sides of the wall, while replacing the isotropic Gaussian kernel with a direction-constrained Gaussian kernel, i.e., the kernel principal axis direction is forced to align with the wall normal;

[0026] S3, two independent texture mapping channels are established for the walls on both sides of the right angle component, and a texture boundary checking module is added in GPU parallel calculation, when the Gaussian kernel coverage region crosses the walls on both sides, the Gaussian kernel is automatically divided into two independent sub-kernels and bound to the corresponding wall texture, and a texture weight zeroing mechanism is implemented for the overlapping area of the sub-kernels;

[0027] S4, a right angle coplanarity loss function based on projection geometry is introduced to detect the coplanar deviation of the Gaussian kernel center points of the walls on both sides of the right angle in real time, and if the included angle between the normal vectors of adjacent Gaussian kernels deviates from the preset angle range, an iterative kernel position correction is triggered until the right angle constraint is met;

[0028] S5, auxiliary Gaussian kernels are implanted within a fixed range of the component edge, and the spatial coordinates of the auxiliary kernels are optimized through edge pixel back-projection, and finally a reconstructed model satisfying the right angle deviation and edge resolution is output.

[0029] The specific steps of identifying the right angle component region in the input image through the deep learning model include:

[0030] A dual-channel edge gradient collaborative perception network is constructed, the first branch of which uses an adaptive window convolution kernel to extract pixel-level right angle topological features, and the window size matches the texture scale of the building, the second branch separates the wall body from the attached objects through a multi-spectral reflectance analysis module, and outputs an edge heat map, finally the feature tensors output by the two branches and the original RGB image are spliced through a channel through a geometric mutation feature fusioner, and a structured semantic label containing internal angle ninety degrees and external angle ninety degrees and edge entity thickness is generated through a three-dimensional convolution layer.

[0031] The specific steps of outputting the semantic segmentation map with geometric mutation feature labels include:

[0032] Based on the structured semantic label, spatial coordinate precision reinforcement coding is performed on the right angle corner coordinates, which are mapped into a high-dimensional feature vector and stored in a geometric mutation feature tensor, which contains the three-dimensional error ellipsoid parameters of the corner spatial coordinates, the cosine similarity threshold of the wall normal vectors on both sides, and the probability distribution histogram of the edge entity thickness, and finally the semantic segmentation map with geometric mutation feature labels is output.

[0033] Prohibiting Gaussian kernel distribution within a radius range of the right angle corner center specifically includes:

[0034] The adaptive coefficient of the forbidden radius is calculated according to the error ellipsoid parameter in the geometric mutation feature mark, when the edge entity thickness probability distribution histogram shows that the thickness reaches the lower limit, the adaptive coefficient of the forbidden radius is reduced in linear proportion, and meanwhile, the ellipsoid type space expansion is implemented on the forbidden area in combination with the cosine similarity deviation of the normal vectors of the two side walls, so that the extension distance of the forbidden area along the perpendicular bisector direction is greater than the perpendicular wall surface direction.

[0035] The gradient attenuation kernel density strategy specifically includes:

[0036] Based on the determined forbidden area boundary, a kernel density attenuation coordinate system is established along the wall surface normal direction, the negative exponential type density attenuation curve is used to control the Gaussian kernel distribution density with the inflection point forbidden area outer edge as the origin, the attenuation rate of the curve is strongly related to the edge entity thickness probability distribution, that is, when the thickness reaches the lower limit, the attenuation rate is increased to enhance the edge sharpness, and meanwhile, a wall surface flatness compensation factor is introduced, if the wall surface is detected to have a slight curvature, the attenuation rate is automatically reduced to avoid the kernel distribution fault.

[0037] The generation of the direction-constrained Gaussian kernel specifically includes:

[0038] According to the wall surface normal vector data in the geometric mutation feature mark, the isotropic Gaussian kernel is reconstructed into an ellipsoid type anisotropic kernel, the main axis direction is forcibly locked to the normal vector direction, the long axis radius is compressed according to the kernel density attenuation curve, and the short axis radius is proportional to the extracted edge thermal map confidence, when the edge thermal map shows that the texture edge is clear, the short axis radius is expanded to improve the texture continuity.

[0039] The operation of automatically segmenting the Gaussian kernel into two independent sub-kernels specifically includes:

[0040] When it is detected that the Gaussian kernel coverage area crosses the texture boundary defined by the semantic segmentation map, the kernel voxel is dissected with the boundary plane as the segmentation surface to generate sub-kernels respectively belonging to the two side walls;

[0041] The implementation criterion of the texture weight zeroing mechanism is:

[0042] If the spatial overlap rate of the sub-kernel with the original kernel after segmentation exceeds the current bit position kernel density threshold, the texture mixing weight of the overlap area is reduced to zero, and the zeroing area width is adjusted according to the short axis radius of the ellipsoid type anisotropic kernel.

[0043] The construction method of the right-angle coplanarity loss function is:

[0044] The direction vector of the main axis of the ellipsoid type anisotropic kernel is calculated in real time. If the two adjacent kernels are located on the walls on the two sides of a right angle, the modulus of the cross product of the main axis vectors of the two kernels is compared with a preset right angle reference value. The coplanar deviation detection is realized by a projection geometry coplanarity checker. The centers of the two side Gaussian kernels are projected to the theoretical plane of the right angle with the center of the inflection point forbidden area as the projection origin. When the angle between the connecting line of the projection points and the theoretical right angle edge deviates from the threshold value, the iterative kernel position migration based on the kernel density attenuation coordinate system is triggered.

[0045] The trigger condition for implanting an auxiliary Gaussian kernel is:

[0046] When the edge heat map confidence of the component edge in the semantic segmentation map is lower than a set threshold value, and the site is marked as a high deviation risk area in the projection geometry coplanarity check, the implanting procedure is started;

[0047] The spatial coordinate optimization of the auxiliary kernel adopts edge pixel back projection closed loop calibration, including inverse projection of the segmented sub-kernel texture boundary coordinates to the input image, calculation of the color gradient residual error between the edge pixels and the projection position, adjustment of the three-dimensional coordinates of the auxiliary kernel by the short axis radius of the ellipsoid type anisotropic kernel, and convergence of the residual error.

[0048] It needs to be further explained and described that in the 3D Gaussian sputtering based building three-dimensional reconstruction, the accurate recognition of the right angle component is the basis for the subsequent kernel distribution and geometric constraint. The traditional single channel network is easy to be disturbed by the wall attachments or the texture scale mismatch, resulting in blurred right angle profile and boundary misjudgment. Therefore, a dual channel edge gradient collaborative perception network needs to be constructed to simultaneously attack from two dimensions of topological feature extraction and separation of the body and attachments, and accurately lock the right angle component area. The specific implementation is as follows:

[0049] The core of constructing the dual channel edge gradient collaborative perception network is to realize feature complementation. The network as a whole is based on the Encoder and Decoder architecture. The input is a building RGB image with a resolution of 256x256. The Encoder part includes 6 convolution blocks. Each convolution block is composed of 2 convolution layers, 2 BatchNorm layers and 2 ReLU activation functions. The Decoder realizes feature up-sampling through transposed convolution. Finally, a feature map with the same size as the input is output. The two channels are embedded in parallel between the 2nd and 5th convolution blocks of the Encoder. The first branch focuses on the extraction of right angle topological features, and the second branch focuses on the separation of the wall body and attachments. The features of the two branches are fused in the Decoder stage to ensure that both the geometric structure and the interference factors are captured.

[0050] During network training, a building right-angle component labeling data set is adopted, containing 5000 building images of different scenes and different illuminations, and the internal angle 90°, external angle 90° and edge thickness are labeled, and the cross entropy loss function + Dice loss function is combined to optimize the recognition accuracy of small targets. The first branch adopts adaptive window convolution kernel to extract pixel-level right-angle topological features. The adaptive window convolution kernel refers to a convolution kernel whose window number and size can be dynamically adjusted according to the texture scale of the input image. It is different from the traditional convolution kernel with fixed window. First, the network calculates the texture period of the input image through the preposed texture analysis module, and after the image is processed by grayscale, the texture frequency is extracted by Fourier transform. The inverse of the frequency is the texture period. Then, the window size is dynamically adjusted according to the texture period. If the texture period is 20 pixels, the window size is set to 10*10 pixels, which ensures that the window can completely cover a single texture unit. If the texture period is 5 pixels, the window size is set to 3*3 pixels, which avoids the loss of details caused by too large window size. At the same time, the window number is fixed to 4*4, forming 16 independent sub-windows, each of which is responsible for capturing the gradient relationship of local pixels. When extracting pixel-level right-angle topological features, each sub-window calculates the gradient direction and gradient amplitude of the pixels in the window, selects the pixel pairs with gradient direction close to 90°, and marks them as candidate right-angle pixels. Then, through the 3*3 neighborhood voting mechanism, if there are at least 3 candidate right-angle pixels in the 8 neighborhoods of a pixel, and the gradient amplitude sum exceeds the preset threshold, it is determined that the pixel is a right-angle topological feature pixel. Finally, a single-channel right-angle topological feature map is output, and the higher the pixel value in the map represents the more significant the right-angle feature.

[0051] The second branch separates the wall body and the attached object through a multispectral reflectance analysis module. The multispectral reflectance analysis module is a functional module that separates the wall and the attached object by utilizing the reflectance difference of different materials in the multispectral band. First, the module generates a near-infrared band image by a spectral conversion algorithm, forming 4-band multispectral data. Then, the multispectral reflectance vector of each pixel is calculated, and the reflectance value of the 4 bands is calculated. The reflectance value is equal to the pixel value divided by the incident light intensity. The incident light intensity is calibrated by the image gray mean value. Next, a preset wall reflectance template library is called. The template library contains the multispectral reflectance vectors of common wall materials such as concrete, ceramic tile, and paint, which are obtained through laboratory spectral measurement. The cosine similarity is used to calculate the matching degree of the current pixel reflectance vector and each vector in the template library. The pixels with a matching degree greater than or equal to 0.85 are marked as wall candidate pixels, and the pixels with a matching degree less than 0.85 are marked as attached object candidate pixels. To further optimize the separation effect, the module introduces a spatial continuity constraint. If more than or equal to 6 of the 8 neighborhoods of a certain attached object candidate pixel are wall candidate pixels, it is determined as a misjudgment and corrected as a wall candidate pixel to avoid small area texture noise being misjudged as an attached object. Finally, the module outputs a single-channel edge heat map. The pixel value (0-255) in the heat map represents the confidence of the right-angle component edge. The pixel value of the boundary between the wall and the attached object, the right-angle boundary of the wall itself, is 200-255 (high confidence), and the pixel value of the inside of the wall and the inside of the attached object is 0-50 (low confidence). The edge position of the right-angle component is intuitively presented. Finally, the geometric mutation feature fusioner is used to channel splice the feature tensors output by the two branches and the original RGB image. The geometric mutation feature fusioner is a module that realizes efficient fusion of multi-source features. Its core logic is channel complementation and weight self-adaptation. First, the right-angle topology feature tensor (1 channel) output by the first branch, the edge heat map feature tensor (1 channel) output by the second branch, and the original RGB image (3 channels) are spliced according to the channel dimension to form a 5-channel fusion feature tensor. Then, the fusioner performs dimension compression and weight distribution on the 5-channel features through a 1x1 convolutional layer. The right-angle topology feature channel and the edge heat map channel are given higher weights (weight coefficients are 1.2-1.5), and the RGB image channel is given basic weights (weight coefficients are 1.0) to ensure that geometric features and edge features are given priority. Finally, the weighted fusion feature tensor is input into a three-dimensional convolutional layer with a convolution kernel size of 3x3x5, a step of 1, and padding of 1. The three-dimensional convolutional layer captures the associated features between channels to generate structured semantic labels. The labeling result is a 3-channel feature map. The first channel labels the internal angle , and the pixel value 1 represents an internal angle, and 0 represents a non-internal angle. The second channel labels the external angle , pixel value 1 represents an external corner, 0 represents a non-external corner, the third channel marks the edge entity thickness, and the pixel value is a normalized value of the thickness in millimeters. The three channels cooperatively constitute a complete right-angle component structured description, laying a foundation for subsequent geometric mutation feature extraction.

[0052] After obtaining the structured semantic annotation, only relying on pixel-level annotation cannot meet the accuracy requirements of subsequent 3D Gaussian kernel distribution, and the coordinate accuracy of the right-angle corner needs to be further strengthened, and geometric constraint parameters need to be supplemented, so spatial coordinate accuracy strengthening coding based on structured semantic annotation is implemented to construct a geometric mutation feature tensor, and finally a semantic segmentation map with feature labels is output. The specific implementation is as follows:

[0053] Based on the structured semantic annotation, first, spatial coordinate accuracy strengthening coding is implemented on the right-angle corner coordinates. The right-angle corner coordinates refer to the center coordinates (x, y) of the 90° internal angle or 90° external angle pixels in the structured semantic annotation. The original coordinates are directly converted from the pixel index. For example, the coordinates of the pixel in the i-th row and the j-th column are (j*pixel spacing, i*pixel spacing, pixel spacing). The pixel spacing is calibrated according to the image resolution and the actual scene scale, such as 1 pixel corresponding to 5 mm. The spatial coordinate accuracy strengthening coding is a process of correcting the original coordinate error through neighborhood geometric constraints:

[0054] First, select 3x3 neighborhood pixels centered on the original inflection point coordinates, extract the gradient direction of the right-angle topological feature pixels in the neighborhood, and fit the straight line equations on both sides of the right angle through the least squares method, such as horizontal straight line y=a, vertical straight line x=b, and the intersection of the straight lines is the corrected inflection point coordinates (b, a), the correction accuracy can reach within 1 pixel, then map the corrected inflection point three-dimensional coordinates (x, y, 0, the initial z coordinate is set to 0, and the subsequent 3D reconstruction is supplemented), coordinate correction amount (the difference between the original coordinates and the corrected coordinates), and neighborhood gradient statistics (variance and mean of gradient direction in the neighborhood) information into a 64-dimensional high-dimensional feature vector, the mapping process is realized through a fully connected layer, the input is 10-dimensional basic information, and the output is a 64-dimensional feature vector, ensuring that the geometric information of the inflection point is fully encoded, finally, store the high-dimensional feature vector into the geometric mutation feature tensor, the geometric mutation feature tensor is a tensor with dimensions HxWx64, H and W are the height and width of the image, only the tensor elements at the right-angle inflection point position are the above-mentioned 64-dimensional feature vectors, and the rest are zero vectors, which efficiently stores the key geometric information, the geometric mutation feature tensor contains three types of core geometric parameters, which comprehensively support the subsequent Gaussian kernel distribution strategy, the first type is the three-dimensional error ellipsoid parameter of the inflection point spatial coordinates, the three-dimensional error ellipsoid is a geometric model for describing the measurement error of the inflection point coordinates, its parameters include long semi-axis, short semi-axis, flattening and spatial orientation, the long semi-axis and the short semi-axis correspond to the error sizes in x and y directions respectively, the flattening reflects the anisotropy degree of the error, flattening=long semi-axis / short semi-axis, the closer the flattening to 1, the more uniform the error, and the spatial orientation is consistent with the normal vector of the two sides of the right angle, which is obtained by fitting the error distribution of the neighborhood pixels through the least squares method, and directly presents the reliability of the inflection point coordinates, the second type is the cosine similarity threshold of the two side wall surface normal vectors, the wall surface normal vector is a unit vector perpendicular to the wall surface, which is calculated by fitting the pixel gradient direction of the wall surface region in the structured semantic annotation, and the cosine similarity threshold is a key indicator for judging whether the two walls are perpendicular, the cosine similarity of the two side wall surface normal vectors of the right-angle component is close to 0, i.e. the included angle is 90°, and the threshold is determined by statistically analyzing the normal vector similarity of a large number of standard right-angle components, which is usually set to 0.1, and the similarity less than or equal to 0.1 is determined as perpendicular, and the similarity greater than 0.1 is determined as non-perpendicular, and the third type is the probability distribution histogram of the edge entity thickness, which is the actual physical thickness of the edge of the right-angle component, which is extracted through the third channel of the structured semantic annotation, the horizontal coordinate of the probability distribution histogram is the thickness interval, and the vertical coordinate is the pixel proportion in the corresponding interval, and the peak interval is the most likely value of the edge entity thickness, which provides a basis for the subsequent forbidden cloth area radius calculation.

[0055] Finally, output the semantic segmentation map with geometric mutation feature markers, the specific process is as follows:

[0056] Firstly, the original RGB image is taken as the base, and the three channels of structured semantic annotation, i.e. interior angle, exterior angle and edge thickness, are generated into a basic semantic segmentation image in the form of pseudo-color superposition. The interior angle region is marked in red, the exterior angle region is marked in blue, and the edge thickness is distinguished by color depth. Subsequently, the core parameters in the geometric mutation feature tensor, including the long semi-axis of the three-dimensional error ellipsoid, the normal vector cosine similarity threshold and the edge thickness peak interval, are embedded into the pixel attributes of the basic semantic segmentation image in the form of feature labels. The attribute field of each pixel contains coordinates, annotation type and geometric parameters. Finally, the semantic segmentation image with attributes is encoded and stored through an image compression algorithm. The binary data of geometric parameters are preserved during the encoding process to ensure that the subsequent adaptive kernel distribution control unit can quickly parse and call. This semantic segmentation image not only visually presents the spatial position and type of the right-angle component, but also implicitly contains high-precision geometric constraint parameters, providing accurate input basis for the subsequent non-uniform Gaussian kernel distribution strategy.

[0057] After obtaining the semantic segmentation image with geometric mutation feature labels, in order to avoid excessive aggregation of the Gaussian kernel at the right-angle corner, resulting in blurred reconstructed edges, a forbidden area needs to be set within the radius range of the corner center. The size and shape of the forbidden area need to be dynamically adjusted in relation to the geometric accuracy of the corner, the edge thickness and the wall normal vector, rather than using a fixed radius. The specific implementation is as follows:

[0058] The forbidden radius adaptive coefficient is calculated according to the error ellipsoid parameters in the geometric mutation feature labels. The error ellipsoid parameters are three-dimensional geometric parameters that describe the coordinate measurement error of the right-angle corner, including the long semi-axis (a), the short semi-axis (b) and the flattening (a / b). The long semi-axis and the short semi-axis correspond to the error size of the corner in the wall normal direction and the tangent direction, respectively. The forbidden radius adaptive coefficient is a parameter used to dynamically adjust the basic radius of the forbidden area. Its calculation logic is as follows:

[0059] First, take the average of the long semi-axis and the short semi-axis of the error ellipsoid, then multiply it by the basic correction factor to get the initial coefficient. If the error ellipsoid flattening is greater than the initial coefficient, it means that the error in one direction is significantly larger. Then multiply it by the flattening correction factor to avoid excessive expansion of the forbidden area caused by too large a coefficient. Finally, the forbidden radius adaptive coefficient is determined. For example, if the error ellipsoid of a corner is a=2.5mm, b=1mm and flattening=2.5, the initial coefficient is 1.2×(2.5+1) / 2=2.1mm, and the coefficient after superimposing the flattening correction is 2.1×0.9=1.89mm. This coefficient directly determines the basic size of the forbidden area. The larger the error, the larger the coefficient, ensuring that the forbidden area can cover the possible error area and avoid the kernel distribution invading the unreliable range of the corner accuracy.

[0060] When the edge entity thickness probability distribution histogram shows that the thickness reaches the lower limit, the forbidden radius adaptive coefficient is reduced in linear proportion, and the edge entity thickness probability distribution histogram is a graph of the statistical right-angle component edge thickness probability distribution, the horizontal coordinate is the thickness interval, and the vertical coordinate is the pixel ratio of the corresponding interval, and the lower limit refers to the minimum thickness value of the peak interval in the histogram, which reflects the minimum physical thickness of the edge entity, and the thinner the thickness, the more delicate the structure at the inflection point, and the forbidden zone needs to be reduced to avoid excessive forbidden to cause insufficient edge kernel distribution, and the specific reduction process is as follows: first, set the target coefficient corresponding to the thickness lower limit based on the basic coefficient, then calculate the reduction ratio according to the difference between the current thickness and the lower limit, if the current thickness is exactly equal to the lower limit, the coefficient is directly adjusted to the target coefficient 1.2mm, if the current thickness is 15mm, the coefficient is calculated as basic coefficient-(basic coefficient-target coefficient)×(lower limit / current thickness), such as 1.89-(1.89-1.2)×(10 / 15)=1.89-0.46=1.43mm, to ensure that the coefficient is reduced linearly with the thickness, and adapts to edge structures of different fineness, and at the same time, the forbidden zone is implemented ellipsoidal space expansion combined with the cosine similarity deviation of the normal vectors of the two side walls, and the cosine similarity deviation of the normal vectors of the two side walls refers to the difference between the actual measured cosine similarity of the normal vectors of the two side walls and the cosine similarity corresponding to the ideal right angle, and the greater the deviation, the more serious the actual angle deviation of the wall surface from the right angle, and the forbidden zone needs to be expanded in the deviation direction to cover the angle deviation area, and when calculating the deviation, first extract the normal vectors of the two side walls from the geometric mutation feature tensor (such as wall surface 1 normal vector n1=(x1, y1, z1), wall surface 2 normal vector n2=(x2, y2, z2)), calculate the cosine similarity (cosθ=|n1·n2| / (|n1|×|n2|)) of the two, and then the difference between the ideal value 0 is the deviation (such as cosθ=0.1, deviation=0.1).

[0061] When the ellipsoidal expansion is performed, the inflection point center is taken as the ellipsoidal center, the forbidden zone is designed as a three-axis ellipsoid, the half-axis length in the right angle bisector direction=forbidden radius adaptive coefficient×(1+deviation×5) (the greater the deviation, the longer the half-axis, such as deviation 0.1, half-axis length=1.2×(1+0.1×5)=1.8mm), and the half-axis length in the direction perpendicular to the two side walls=forbidden radius adaptive coefficient (such as 1.2mm), and this design makes the extension distance of the forbidden zone in the right angle bisector direction (1.8mm) greater than that in the direction perpendicular to the wall (1.2mm), which can cover the unstable area caused by the angle deviation, and also will not excessively occupy the effective space of the wall, and ensures that the subsequent Gaussian kernel is only distributed in the stable area of the wall.

[0062] After determining the ellipsoidal forbidden area of the right-angle inflection point, the Gaussian kernel distribution density of the wall surface on both sides of the inflection point needs to be gradient controlled. If uniform kernel distribution is adopted, it is easy to cause insufficient kernel density near the edge of the forbidden area or excessive kernel density far from the forbidden area. Therefore, a kernel density attenuation coordinate system needs to be established based on the boundary of the forbidden area, and a negative exponential curve is used to realize density gradient attenuation, while adapting to the edge thickness and wall flatness. The specific implementation is as follows:

[0063] Based on the determined forbidden area boundary, a kernel density attenuation coordinate system is established along the wall normal direction. The forbidden area boundary refers to the outer surface of the ellipsoidal forbidden area, which can be calculated by the error ellipsoid parameter and the expansion coefficient in the geometric mutation feature tensor. The wall normal direction is the direction perpendicular to the wall plane and pointing to the outside of the wall. The wall normal vector needs to be extracted from the geometric mutation feature tensor, such as the wall normal vector n=(0,0,1) representing the wall perpendicular to the z-axis, and the normal direction along the positive direction of the z-axis. The coordinate system can accurately reflect the distance relationship between the kernel and the forbidden area. When establishing the coordinate system, the point on the outer edge of the inflection point forbidden area closest to the wall is taken as the origin (O point). The x-axis is set to point outward along the wall normal direction, away from the forbidden area. The y-axis is along the tangent direction of the wall, parallel to the edge of the wall. The z-axis is along the other tangent direction of the wall, perpendicular to the y-axis. A three-dimensional rectangular coordinate system is formed. The coordinates (x, y, z) of each point in the coordinate system directly correspond to the spatial position of the kernel. The x value represents the distance between the kernel and the outer edge of the forbidden area. x=0 is the origin, x>0 is away from the forbidden area, and x<0 is in the forbidden area. This provides an intuitive distance basis for subsequent density attenuation calculation.

[0064] A negative exponential density attenuation curve is used to control the Gaussian kernel distribution density. The negative exponential density attenuation curve is a curve in which the kernel density gradually decreases according to a negative exponential law as the x value increases. Its core advantage is to maintain a high kernel density near the forbidden area to ensure clear edge details, and gradually reduce the density away from the forbidden area to avoid kernel redundancy. The specific implementation process is as follows:

[0065] First, set the initial density (density at x=0, such as 5 kernels per cubic millimeter) and the attenuation coefficient (preset as 0.2, controlling the attenuation speed). The density calculation formula is: current density = initial density x e^(-x / attenuation coefficient), where e is the natural constant. This gradient distribution ensures that the edge region has enough kernel support for detail reconstruction, and avoids excessive kernel density in the distance to increase the computational burden.

[0066] The curve attenuation rate is strongly related to the edge entity thickness probability distribution, i.e. when the thickness reaches the lower limit, the attenuation rate increases to enhance the edge sharpness. The attenuation rate refers to the attenuation coefficient in the negative exponential curve that controls the density decline speed. The thinner the edge entity thickness, the finer the wall edge, and the attenuation rate needs to be increased to quickly concentrate the kernel density near the forbidden area, enhancing the sharpness of the edge and avoiding edge blur. The specific correlation process is as follows:​

[0067] The peak interval of the current thickness is extracted from the edge entity thickness probability distribution histogram, and the mapping relationship between the thickness and the attenuation rate is set. When the thickness reaches the lower limit, the attenuation rate is increased to 0.3, which is higher than the basic attenuation rate 0.2. At this time, the density at x=0 is still 5 / mm³, and the density at x=2mm is 5×e^(-0.3×2)=5×0.55=2.75 / mm³. The density decreases faster, and the nuclei near the forbidden area are more concentrated. When the thickness is 15mm (normal thickness), the attenuation rate maintains the basic value 0.2. When the thickness is 20mm (thick edge), the attenuation rate is reduced to 0.15, and the density decreases more slowly, ensuring that the thick edge area has enough nuclei to cover a larger area, balancing the nuclear distribution requirements of different thickness edges. At the same time, a wall flatness compensation factor is introduced. If the wall is detected to have a slight curvature, the attenuation rate is automatically reduced to avoid nuclear distribution faults. The wall flatness compensation factor is a parameter used to modify the attenuation rate. When the wall has a slight curvature, if the original attenuation rate is still used, it may cause the nuclear density in the curved area to be too low to form a fault, and the attenuation rate needs to be reduced to slow down the density decrease. The specific implementation process is as follows:

[0068] First, detect the wall flatness. Extract the pixel three-dimensional coordinates (x, y, z) of the wall area from the semantic segmentation map. Select a 10x10mm neighborhood pixel. Fit the plane equation by least squares method. Calculate the deviation of each pixel z coordinate in the neighborhood from the fitted plane. If the maximum deviation is greater than 0.5mm, it is determined that there is a slight curvature, and the compensation mechanism is started. When compensating, the wall flatness compensation factor is set to 0.85, which is less than 1. Multiply the current attenuation rate by the compensation factor to get the modified attenuation rate. After modification, the density at x=2mm is 5×e^(-0.255×2)=5×0.59=2.95 / mm³. The density decreases more slowly, avoiding the sudden drop in nuclear density caused by the rapid increase of x value in the curved area, and ensuring the overall nuclear distribution of the wall to be continuous and fault-free.

[0069] After completing the design of the nuclear density gradient distribution, the traditional isotropic Gaussian kernel (spherical kernel) cannot adapt to the planar structure of the wall. The coverage range of the spherical kernel in the normal direction and the tangent direction of the wall is consistent, which easily leads to overextension of the normal direction nuclear or insufficient coverage in the tangent direction. Therefore, it is necessary to generate a direction-constrained Gaussian kernel according to the wall normal vector, i.e. an ellipsoidal anisotropic kernel, so that the shape of the kernel is accurately matched with the wall structure. The specific implementation method is as follows:

[0070] According to the wall normal vector data in the geometric mutation feature marker, the isotropic Gaussian kernel is reconstructed into an ellipsoidal anisotropic kernel. The isotropic Gaussian kernel refers to a spherical kernel with the same radius in all directions in space. The ellipsoidal anisotropic kernel refers to an ellipsoidal kernel with different radii in different directions, and its shape is determined by the wall normal vector. The core is to make the kernel cover wider in the tangent direction of the wall to adapt to the planar extension of the wall, and more compact in the normal direction to avoid exceeding the wall thickness. The specific reconstruction process is as follows:

[0071] First, the normal vector of the current wall surface is extracted from the geometric mutation feature tensor, such as n=(0,0,1), which represents that the wall surface is perpendicular to the z-axis, the normal direction is along the z-axis, and the three principal axis directions of the ellipsoid are determined. The long axis direction is forced to be consistent with the normal vector direction, the z-axis, and the two short axis directions are along the tangent direction of the wall surface, the x-axis and the y-axis, which are perpendicular to the normal vector. Then the radius of the spherical core is decomposed into the long axis radius and the short axis radius, completing the reconstruction from a sphere to an ellipsoid. For example, the original spherical core radius is 1mm, the long axis radius after reconstruction is set to 0.8mm along the normal direction, which is more compact, and the short axis radius is set to 1.2mm along the tangent direction, which is more extensive, forming an ellipsoid-shaped anisotropic core that adapts to the plane structure of the wall surface, ensuring that the core is only distributed in the effective area of the wall surface and does not excessively extend in the normal direction, and the principal axis direction is forced to lock to the normal vector direction. The long axis radius is compressed according to the core density decay curve, and the short axis radius is proportional to the edge thermal map confidence. The forced locking of the principal axis direction is achieved through the direction matrix of the core. The long axis direction vector of the ellipsoid-shaped anisotropic core is set to the unit vector of the wall surface normal vector, such as the normal vector n=(0,0,1), and the long axis direction vector is (0,0,1). During the core generation process, regardless of the position of the wall surface, the long axis direction always remains consistent with the normal vector, ensuring that the direction of the core is strictly aligned with the structure of the wall surface. The long axis radius is compressed according to the core density decay curve, which means that the long axis radius decreases with the increase of x value (distance from the forbidden area) according to the decay curve. The density decay ratio corresponding to the current x value is extracted from the core density decay curve, and the current long axis radius is obtained by multiplying the ratio. For example, when x=2mm, the long axis radius=0.8×0.67=0.536mm, making the core more compact in areas far from the forbidden area and adapting to the requirement of reduced density. The short axis radius is proportional to the edge thermal map confidence, where the edge thermal map confidence refers to the pixel value (0-255) of the current core position in the edge thermal map output by the second branch. The higher the confidence, the clearer the right-angle edge at that position, and the short axis radius needs to be increased to improve texture continuity. The specific proportional relationship is:

[0072] The confidence of 255 (highest) corresponds to a short axis radius of 1.5mm, and the confidence of 0 corresponds to a short axis radius of 0.8mm. The current short axis radius=0.8+(1.5-0.8)×(current confidence / 255), for example, the current confidence is 200, the short axis radius=0.8+0.7×(200 / 255)=0.8+0.55=1.35mm, ensuring that the core in the clear edge area can better cover the texture details. When the edge thermal map shows that the texture edge is clear, the short axis radius is expanded to improve the texture continuity. The judgment standard for clear texture edge is edge thermal map confidence≥200 (high confidence interval). This type of area is mostly the boundary between the wall surface and the attached object, or the clear outline of right-angle components. The short axis radius needs to be expanded to make the coverage range of adjacent cores overlap more, avoiding texture breakage. The specific expansion process is:

[0073] First, set the confidence threshold (200), if the confidence of the current core position is greater than or equal to 200, then increase the expansion coefficient (0.2-0.3) on the basis of the original short axis radius, for example, the original short axis radius is 1.35mm, and after expansion =1.35+0.25=1.6mm; At the same time, the center distance of adjacent cores is set as 0.8 times the short axis radius (such as radius 1.6mm, distance =1.6x0.8=1.28mm), to ensure that the coverage overlap rate of adjacent cores in the tangent direction is greater than or equal to 20%, forming a continuous texture coverage, if the confidence is less than 200 (texture blur area), the short axis radius is not expanded, only the basic value is maintained, balance the texture continuity and core distribution efficiency, through this direction constraint and radius dynamic adjustment, the generated Gaussian kernel can not only adapt to the plane structure of the wall, but also accurately respond to the texture clarity, which provides guarantee for the geometric precision and texture quality of subsequent 3D reconstruction.

[0074] The specific implementation of Gaussian kernel segmentation, right-angle coplanarity loss function construction and auxiliary Gaussian kernel implantation is as follows:

[0075] When it is detected that the Gaussian kernel coverage area crosses the texture boundary defined by the semantic segmentation map, the definition of the texture boundary needs to be determined first. The boundary is the boundary line of different material areas in the semantic segmentation map, and the edge pixel value is marked with the boundary normal vector, which is perpendicular to the boundary tangent direction. The specific process of detection is as follows:

[0076] From the GPU computing pipeline of the parallel texture processing unit, the spatial bounding box parameters of the current Gaussian kernel are called, the center of the ellipsoidal anisotropic kernel is taken as the origin, the long axis and short axis radii are determined, the xmin / xmax, ymin / ymax, zmin / zmax range, and the texture boundary three-dimensional coordinates in the corresponding spatial range are extracted from the semantic segmentation map. Through the mapping relationship conversion of the pixel coordinates and the 3D space coordinates, if the bounding box of the Gaussian kernel intersects with the texture boundary, that is, the boundary line passes through the inside of the bounding box, it is determined that the kernel crosses the texture boundary. At this time, kernel voxel dissection is implemented with the boundary method plane as the partition surface. The boundary method plane refers to the plane that passes through the texture boundary and has the same normal vector as the boundary normal vector. The plane equation is obtained by fitting the 3D coordinates of three consecutive pixels on the boundary. Kernel voxel dissection is to discretize the ellipsoidal anisotropic kernel into 0.1mmx0.1mmx0.1mm micro voxel units. Each voxel unit is judged to belong to the region by substituting the plane equation. If Ax+By+Cz+D<0, the voxel belongs to the wall surface A on one side of the texture boundary. If Ax+By+Cz+D>0, the voxel belongs to the wall surface B on the other side. After dissection, the same side voxels are re-aggregated to generate two independent sub-kernels, sub-kernel A and sub-kernel B. Each sub-kernel retains the direction parameters of the original kernel, including the main axis direction and the long / short axis radius. Only the volume is proportionally reduced according to the number of voxels. At the same time, the corresponding wall surface texture ID is respectively bound, such as sub-kernel A binding the texture data of wall surface A and sub-kernel B binding the texture data of wall surface B, to ensure that the sub-kernel and the texture of the wall surface are strictly matched.

[0077] The implementation of the texture weight zero mechanism needs to avoid texture mixing in the overlapping area of the sub-kernels. The criterion revolves around the spatial overlap rate and the kernel density threshold. The spatial overlap rate refers to the proportion of the number of voxels overlapped by the segmented sub-kernel and the original Gaussian kernel to the total number of voxels of the original kernel. For example, if the original kernel contains 1000 voxels, sub-kernel A overlaps 800 voxels with the original kernel, and the overlap rate is 80%. The current site kernel density threshold is the upper limit of the Gaussian kernel distribution density at this spatial position. For example, based on the gradient decay curve, the current site density threshold is 5 kernels per cubic millimeter, and the corresponding overlap rate threshold is 30%. That is, an overlap rate exceeding 30% is likely to cause excessive texture mixing. The specific implementation process is as follows:

[0078] First, the spatial overlap rate of each sub-kernel and the original kernel is calculated. If the overlap rate of a certain sub-kernel exceeds the current site kernel density threshold, the texture mixing weight of the overlapping area in the sub-kernel is reduced to zero. The texture mixing weight is a parameter that controls the contribution of the sub-kernel texture to the reconstructed model. A weight of 1 represents complete contribution, and a weight of 0 represents no contribution. After the weight of the overlapping area is cleared, the area is only contributed by the texture of the sub-kernel on the other side, avoiding the blurring of the mixed texture. At the same time, the width of the cleared area is adjusted according to the short axis radius of the ellipsoidal anisotropic kernel. The short axis radius reflects the coverage of the kernel in the tangent direction of the wall. The larger the short axis radius, the wider the cleared area proportionally increases to ensure that the cleared area can completely cover the overlapping area without affecting the texture continuity of the non-overlapping area. For example, for a sub-kernel with a short axis radius of 1.2 mm, when the overlap rate exceeds the threshold, the cleared area width is set to 0.24 mm, which extends along the texture boundary to the interior of the sub-kernel, accurately eliminating the interference of overlapping textures.

[0079] On the basis of ensuring accurate segmentation of Gaussian kernels and reasonable control of texture weights, if there is a coplanar deviation of Gaussian kernels on the two walls of a right angle, it will cause distortion of the reconstructed right-angle component. Therefore, a right-angle coplanarity loss function needs to be constructed to ensure that the spatial position of the kernel meets the right-angle constraint through real-time detection and iterative correction. The specific implementation is as follows:

[0080] The construction of the right-angle coplanarity loss function starts with the real-time calculation of the major axis direction vector of the ellipsoidal anisotropic kernel. The major axis direction vector is the unit vector of the long axis direction of the ellipsoidal anisotropic kernel, which is consistent with the direction of the wall normal vector. Its extraction process is as follows:

[0081] From the kernel parameters output by the adaptive kernel distribution control unit, the long axis direction parameters of each ellipsoidal anisotropic kernel are retrieved. Through unitization processing, i.e., dividing by the vector module length, the major axis direction unit vector is obtained, ensuring that the vector length is uniform as 1, which is convenient for subsequent calculations. Through the wall ID distinction of the semantic segmentation map, if the two adjacent kernels are located on the two walls of a right angle, such as kernel 1 belonging to wall A and kernel 2 belonging to wall B, and the two walls are labeled as a right-angle relationship, then the modulus of the cross product of the two kernel major axis direction vectors is calculated. The modulus of the cross product can reflect the perpendicularity of the two vectors. In an ideal right angle, the normal vectors of the two walls are perpendicular, and the modulus of the cross product = the product of the lengths of the two vectors. Since the vectors are unitized, the length product is 1. The preset right-angle reference value, which is the cross product modulus value when the ideal right angle is 1, is compared with the actual calculated cross product modulus. If the modulus value is less than 0.9 (threshold, representing that the angle between the two vectors deviates from 90° by more than 6°), it is determined that the two kernels have coplanar deviation, triggering the loss function calculation. The loss value = reference value - actual cross product modulus. The larger the loss value, the more serious the deviation.

[0082] The coplanar deviation detection is realized by a projection geometry coplanarity verifier, which is a functional module for verifying whether the kernel center points are coplanar based on the projection geometry principle. First, the projection parameters are determined, and the inflection point forbidden region center is taken as the projection origin. The forbidden region center coordinates are extracted from the geometric catastrophe characteristic tensor. The right-angle theoretical plane refers to a plane constructed based on the ideal geometric shape of the right-angle component. The plane z=Oz is fitted by the inflection point forbidden region center and the ideal right-angle edges of the two side walls. Assuming that the right angle is in the xy plane, the plane represents the geometric plane that the right-angle component should have. The projection process is as follows:

[0083] The Gaussian kernel center points of the two side walls, such as the kernel 1 center point (x1, y1, z1) of wall A and the kernel 2 center point (x2, y2, z2) of wall B, are projected onto the right-angle theoretical plane along the line connecting the projection origin and the kernel center points to obtain the projection points, such as the projection point (x1', y1', Oz) of kernel 1 and the projection point (x2', y2', Oz) of kernel 2. The projection coordinates are calculated based on the similar triangle principle, such as x1'=Ox+(x1-Ox)×(Oz-Oz) / (z1-Oz), which is simplified to x1'=x1 and y1'=y1. Since the theoretical plane is z=Oz, the projection relationship needs to be accurately corrected by combining the camera internal and external parameters during actual projection. When the angle between the projection point line and the theoretical right-angle edge deviates from the threshold value, the iterative kernel position migration based on the kernel density attenuation coordinate system is triggered. The theoretical right-angle edge refers to the ideal right-angle edge in the right-angle theoretical plane, such as the edge along the x-axis as the theoretical edge 1 and the edge along the y-axis as the theoretical edge 2. The angle between the projection point line, such as the line connecting the projection points of kernel 1 and kernel 2, and the theoretical right-angle edge is calculated. If the angle deviates from the preset threshold value, the iterative migration is started. The migration coordinate system uses the kernel density attenuation coordinate system established in the foregoing, the x-axis is along the wall normal direction, the y-axis and the z-axis are along the wall tangent direction, and the migration object is the kernel on the deviation side. The migration step size is determined according to the loss value, and the center coordinates of the kernel are adjusted along the x-axis direction (normal direction). After each migration, the main axis vector cross product module and the projection angle are recalculated. If the angle is less than or equal to the threshold value and the loss value is less than 0.05, the migration is stopped. If the iteration is not up to standard after 5 times, the step size is increased to 0.2 mm for continuous migration until the right-angle constraint is met, ensuring that all kernel center points are in the theoretical plane and eliminating the coplanar deviation.

[0084] After optimizing the kernel position and improving the geometric precision through the right-angle coplanarity loss function, the edge region of the component may have low edge thermal map confidence and high coplanar deviation risk, resulting in sparse Gaussian kernel distribution and affecting the edge resolution of the reconstructed model. Therefore, auxiliary Gaussian kernels need to be implanted and their spatial coordinates need to be optimized to fill the edge detail gap. The specific implementation is as follows:

[0085] The trigger condition for implanting the auxiliary Gaussian kernel needs to meet both the low edge heat map confidence and the high deviation risk area, and one of them is indispensable. The edge heat map confidence is the value (0-255) of the edge pixel in the edge heat map output by the second branch. The threshold is determined by a large number of reconstruction samples. If the value is lower than the threshold, it means that the edge texture is blurred, and the kernel distribution is sparse. If the confidence of a certain site is less than 150, it is marked as a low-confidence edge. The high deviation risk area is the area where the projection angle of the kernel center deviates from the threshold in the projection geometry coplanarity check, but does not reach the iteration migration standard. The checker will mark the area as a high deviation risk label. When a site is marked as a low-confidence edge and a high-deviation risk area at the same time, the auxiliary kernel implantation program is automatically started. The implantation position is the 3D space coordinate corresponding to the edge pixel of the site. Through the pixel and space mapping relationship of the semantic segmentation map, the initial parameters of the auxiliary kernel inherit the direction parameters of the surrounding normal kernel. The main axis direction is consistent with the wall normal vector. The long axis radius is 0.8 times the long axis radius of the surrounding kernel, and the short axis radius is 0.8 times the short axis radius of the surrounding kernel. This ensures that the auxiliary kernel is compatible with the surrounding kernel in shape, avoiding being conspicuous.

[0086] The spatial coordinate optimization of the auxiliary kernel uses edge pixel back projection closed loop calibration. This calibration method makes the texture of the auxiliary kernel accurately match the edge pixels of the input image. First, the segmented sub-kernel texture boundary coordinates are inversely projected to the input image. The sub-kernel texture boundary coordinates are the 3D coordinates of the texture boundary of the segmented sub-kernel around the auxiliary kernel. Then, the color gradient residual error between the edge pixels and the projection position is calculated. The edge pixels are the real edge pixels of the region extracted from the input image, which are obtained by the Canny edge detection algorithm. The color gradient residual error is the total sum of the RGB gradient difference between the edge pixels and the projection position pixels. The short axis radius of the ellipsoidal anisotropic kernel is used to adjust the three-dimensional coordinates of the auxiliary kernel. The short axis radius reflects the coverage range of the auxiliary kernel in the tangent direction of the wall. The larger the short axis radius, the larger the adjustment step. The adjustment direction is determined according to the source of the residual error. If the residual error mainly comes from the x direction, the projection position is to the left, then the center coordinates of the auxiliary kernel are adjusted along the x axis. If the source is in the z direction, the projection position is above, then the center coordinates of the auxiliary kernel are adjusted along the z axis. After each adjustment, the projection position and the residual error are recalculated. If the residual error is less than 0.1 (convergence threshold, which represents that the color matching accuracy meets the standard), the adjustment is stopped. If the residual error does not converge after 10 iterations, the step is increased by 1.2 times for further optimization until the residual error converges. Finally, the spatial coordinates of the auxiliary kernel that accurately match the edge of the input image are obtained, the edge kernel distribution gap is filled, and the edge resolution of the reconstruction model is improved.

[0087] The second object of the present application provides a system for implementing a 3D Gaussian sputtering-based building three-dimensional reconstruction method, comprising:

[0088] The building construction semantic analysis unit 1 integrates a double-channel edge gradient collaborative perception network and a geometric mutation feature fusion device, inputs an original image and outputs a semantic segmentation graph carrying a geometric mutation feature tensor, and the tensor contains an error ellipsoid parameter, a wall surface normal vector threshold and an edge thickness probability distribution histogram;

[0089] The adaptive kernel distribution control unit 2 receives a geometric mutation feature label of the semantic segmentation graph, establishes an ellipsoid space expansion forbidden area at a right-angle corner and calculates a forbidden radius adaptive coefficient, synchronously generates a kernel density decay coordinate system to drive an exponential density decay curve, and simultaneously reconstructs an isotropic Gaussian kernel into an ellipsoid anisotropic kernel with a long-axis radius compression and a short-axis radius expansion according to an edge heat map confidence;

[0090] The parallel texture processing unit 3 deploys a texture boundary checking module in a GPU calculation pipeline, triggers kernel voxel subdivision to generate a sub-kernel binding an independent texture channel when detecting that the ellipsoid anisotropic kernel crosses the semantic segmentation boundary, and implements a texture weight zeroing mechanism based on a comparison between a spatial overlap rate and a kernel density threshold;

[0091] The geometric precision enhancement unit 4 internally has a projection geometry coplanarity checker that calculates an ellipsoid anisotropic kernel principal axis vector cross product module in real time, triggers iterative kernel position migration with the forbidden area center as a projection origin, and simultaneously starts an auxiliary kernel implantation program according to an edge heat map confidence and a coplanar deviation risk, and optimizes auxiliary kernel spatial coordinates through edge pixel back projection closed-loop calibration until color gradient residual error converges.

[0092] The application identifies right-angle components of a building through a double-channel network, outputs a semantic segmentation graph with geometric mutation features, generates a non-uniform Gaussian kernel distribution, contains a right-angle corner ellipsoid forbidden area and a wall surface normal direction gradient decay kernel density, reconstructs an isotropic kernel into an ellipsoid anisotropic kernel aligned with the normal, establishes an independent texture channel, splits a cross-surface Gaussian kernel and zeroes texture weights in an overlap area, introduces a right-angle coplanarity loss function to correct kernel coplanar deviation, implants an auxiliary kernel in an edge low-confidence high-deviation area, optimizes coordinates through back projection calibration, and the system contains a building component semantic analysis unit, an adaptive kernel distribution control unit, a parallel texture processing unit and a geometric precision enhancement unit, finally outputs a high-precision reconstruction model with right-angle non-distortion and high edge resolution, and adapts to building construction drawing restoration requirements.

[0093] The basic principles, main features and advantages of the application are shown and described above. Those skilled in the art should understand that the application is not limited by the above examples, and the above examples and descriptions in the specification are only preferred examples of the application and are not intended to limit the application. Without departing from the spirit and scope of the application, various changes and improvements can be made to the application, and these changes and improvements all fall within the scope of the claimed application. The scope of protection of the application is defined by the appended claims and their equivalents.

Claims

1. A 3D Gaussian sputtering-based building three-dimensional reconstruction method, characterized in that: The method comprises the following steps: S1, identifying the building right-angle component region in the input image through a deep learning model, including the accurate contour boundary of the wall corner and the door and window frame, and outputting a semantic segmentation map with geometric mutation feature markers; a double-channel edge gradient collaborative perception network is constructed, a first branch of which uses an adaptive window convolution kernel to extract pixel-level right-angle topological features, the window size matching the texture scale of the building, a second branch of which separates the wall body from the attached objects through a multispectral reflectance analysis module, and outputs an edge heat map, and finally the feature tensors output by the two branches are spliced with the original RGB image through a geometric mutation feature fusion device, and a three-dimensional convolution layer is used to generate a structured semantic label containing the internal angle of 90 degrees and the external angle of 90 degrees and the edge entity thickness; S2, for the right-angle component region marked in the semantic segmentation map, a non-uniform Gaussian kernel distribution strategy is generated, including prohibiting Gaussian kernel distribution within the radius range of the right-angle corner center and implementing gradient attenuation kernel density strategy in the wall surface extension area along the normal direction on both sides of the corner, and replacing the isotropic Gaussian kernel with a direction-constrained Gaussian kernel, that is, the kernel main axis direction is forced to align with the wall normal direction; the generation of the direction-constrained Gaussian kernel specifically comprises: According to the wall normal vector data in the geometric mutation feature markers, the isotropic Gaussian kernel is reconstructed into an ellipsoidal anisotropic kernel, the main axis direction of which is forced to lock to the normal vector direction, the long axis radius is compressed according to the kernel density attenuation curve, and the short axis radius is proportional to the edge heat map confidence, when the texture edge is clear, the short axis radius is expanded to improve the texture continuity; S3, independent texture mapping channels are established for the walls on both sides of the right-angle component, and a texture boundary checking module is added in GPU parallel calculation, when the Gaussian kernel coverage area crosses the walls on both sides, the Gaussian kernel is automatically divided into two independent sub-kernels and is respectively bound to the corresponding wall texture, and the texture weight zeroing mechanism is implemented in the sub-kernel overlap area; the implementation criterion of the texture weight zeroing mechanism is: If the spatial overlap rate of the sub-kernel after division with the original kernel exceeds the current kernel density threshold, the texture mixing weight of the overlap area is reduced to zero, and the zeroing area width is adjusted according to the short axis radius of the ellipsoidal anisotropic kernel; S4, a right-angle coplanarity loss function based on projection geometry is introduced to detect the coplanar deviation of the Gaussian kernel center points of the walls on both sides of the right-angle in real time, and if the included angle of the normal vectors of adjacent Gaussian kernels deviates from the preset angle range, iterative kernel position correction is triggered until the right-angle constraint is met; S5, auxiliary Gaussian kernels are implanted in the fixed range of the component edge, the spatial coordinates of the auxiliary kernels are optimized through edge pixel back projection, and finally a reconstructed model meeting the right-angle deviation and edge resolution is output.

2. The method according to claim 1, wherein the method is characterized by: The semantic segmentation map with geometric mutation feature markers specifically comprises: Based on the structured semantic annotation, the right angle corner coordinate is subjected to spatial coordinate precision reinforcement coding, is mapped to a high-dimensional feature vector and is stored in a geometric mutation feature tensor, the tensor containing three-dimensional error ellipsoid parameters of the corner space coordinate, a cosine similarity threshold of the two side wall normal vectors and a probability distribution histogram of the edge entity thickness, and finally outputs a semantic segmentation graph with geometric mutation feature markers; wherein the three-dimensional error ellipsoid is a geometric model for describing the corner coordinate measurement error.

3. The method according to claim 1, wherein the method is characterized by: The Gaussian kernel distribution is prohibited within the radius range of the right angle corner center, and specifically includes: An adaptive coefficient of the prohibited radius is calculated according to the error ellipsoid parameters in the geometric mutation feature markers, when the probability distribution histogram of the edge entity thickness shows that the thickness reaches the lower limit, the adaptive coefficient of the prohibited radius is reduced in linear proportion, and at the same time, an ellipsoid type spatial expansion is implemented on the prohibited area in combination with the cosine similarity deviation of the two side wall normal vectors, so that the prohibited area extends in the direction of the right angle bisector to a distance greater than the direction perpendicular to the wall; wherein the ellipsoid type spatial expansion takes the corner center as the ellipsoid center, and designs the prohibited area as a three-axis ellipsoid, the half-axis length in the direction of the right angle bisector = adaptive coefficient of the prohibited radius × (1 + deviation × 5), and the half-axis length in the direction perpendicular to the two side walls = adaptive coefficient of the prohibited radius.

4. The method according to claim 3, wherein the method is characterized by: The gradient attenuation kernel density strategy specifically includes: Based on the determined prohibited area boundary, a kernel density attenuation coordinate system is established along the wall normal direction, taking the outer edge of the corner prohibited area as the origin, and a negative exponential type density attenuation curve is used to control the Gaussian kernel distribution density, when the attenuation rate of the curve is strongly related to the probability distribution histogram of the edge entity thickness, i.e. the attenuation rate is increased to enhance the edge sharpness when the thickness reaches the lower limit, and at the same time, a wall flatness compensation factor is introduced, if the wall is detected to have a slight curvature, the attenuation rate is automatically reduced to avoid the kernel distribution fault; the negative exponential type density attenuation curve refers to a curve in which the kernel density gradually decreases according to the negative exponential law as the x value increases.

5. The method according to claim 4, wherein the method is characterized by: The operation of automatically segmenting the Gaussian kernel into two independent sub-kernels specifically includes: When it is detected that the Gaussian kernel coverage area crosses the texture boundary defined by the semantic segmentation graph, the kernel voxel is dissected with the boundary plane as the dissection surface to generate sub-kernels respectively belonging to the two side walls; the kernel voxel dissection is to disperse the ellipsoid type anisotropic kernel into a micro voxel unit, and each voxel unit is judged to belong to a region by substituting into a plane equation.

6. The method according to claim 4, wherein the method is characterized by: The construction method of the right angle coplanarity loss function is: The main axis direction vector of the ellipsoid type anisotropic kernel is calculated in real time, if the adjacent two kernels are located on the two side walls of the right angle, the modulus of the cross product of the main axis vectors is compared with a preset right angle reference value, wherein the detection of the coplanar deviation is realized by a projection geometry coplanarity checker, taking the center of the corner prohibited area as the projection origin, projecting the center points of the two side Gaussian kernels to the theoretical plane of the right angle, when the included angle between the connecting line of the projection points and the theoretical right angle side deviates from the threshold value, the iterative kernel position migration based on the kernel density attenuation coordinate system is triggered.

7. The method according to claim 4, wherein the method is characterized by: The trigger condition of the implanted auxiliary Gaussian kernel is: When the edge thermal map confidence of the component edge in the semantic segmentation graph is lower than a set threshold value, and the site is marked as a high deviation risk area in the projection geometry coplanarity check, the implantation program is started; The spatial coordinate optimization of the auxiliary kernel adopts edge pixel back-projection closed-loop calibration, including inverse projection of the segmented sub-kernel texture boundary coordinates to the input image, calculation of the color gradient residual error of the edge pixels and the projection position, adjustment of the short-axis radius of the ellipsoidal anisotropic kernel to the three-dimensional coordinates of the auxiliary kernel until the residual error converges.

8. A system for implementing a method for 3D reconstruction of buildings based on 3D Gaussian splotter according to any one of claims 4-7, characterized in that, Comprise: The building construction semantic analysis unit (1) integrates a dual-channel edge gradient collaborative perception network and a geometric mutation feature fusioner, inputs an original image and outputs a semantic segmentation graph carrying a geometric mutation feature tensor, which includes error ellipsoid parameters, wall surface normal vector threshold and edge thickness probability distribution histogram; The adaptive kernel distribution control unit (2) receives the geometric mutation feature label of the semantic segmentation graph, establishes an ellipsoidal space expansion forbidden area at the right-angle corner and calculates the forbidden radius adaptive coefficient, synchronously generates a kernel density attenuation coordinate system to drive an exponential density attenuation curve, and simultaneously reconstructs the isotropic Gaussian kernel into an ellipsoidal anisotropic kernel with compressed long-axis radius and expanded short-axis radius according to the edge heat map confidence; The parallel texture processing unit (3) deploys a texture boundary checking module in the middle of the GPU calculation pipeline, and when it detects that the ellipsoidal anisotropic kernel crosses the semantic segmentation boundary, it triggers kernel voxel dissection to generate sub-kernels bound to independent texture channels, and implements a texture weight zeroing mechanism based on the comparison of spatial overlap rate and kernel density threshold; The geometric accuracy enhancement unit (4) has a built-in projection geometric coplanarity checker that calculates the ellipsoidal anisotropic kernel main axis vector cross product module in real time, triggers iterative kernel position migration with the forbidden area center as the projection origin, and according to the edge heat map confidence and coplanar deviation risk, starts the auxiliary kernel implantation program, and optimizes the spatial coordinates of the auxiliary kernel through edge pixel back-projection closed-loop calibration until the color gradient residual error converges. The color gradient residual error is the total sum of the RGB gradient difference between the edge pixels and the projection position pixels.

Images