A three-dimensional real scene model construction method and system for a sloping wall dam construction site

Abstract

This invention provides a method and system for constructing a 3D real-scene model of a sloping wall dam construction site. The method includes: using an image decoupling convolutional neural network to separate instantaneous shadows from steady-state albedo, generating a shadow-free image; extracting multi-scale allometric features, stitching and fusing them with point-by-point convolutional features to generate an allometric confidence map, and writing it into the point cloud attribute field to form an initial point cloud with misalignment weights; optimizing the initial point cloud using the measured sloping wall filling boundary as a constraint, coupling a BIM model to generate a triangular mesh, and using a graph convolutional network to encode the misalignment of the triangular mesh to generate a mirror twin model and the corresponding 3D real-scene model. This invention eliminates the risk of shadow edges being misjudged as structural edges from the source and achieves automatic snapping of the mirror twin mesh model at the allometric interface, ensuring that the misalignment of the clay and rockfill boundaries is consistent in direction and converges in magnitude, thereby constructing a 3D real-scene model with unified boundaries and improving the accuracy and realism of the model.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology for inclined wall dams, and in particular to a method and system for constructing a 3D real-scene model of an inclined wall dam construction site. Background Technology

[0002] A sloping-wall dam is a type of earth-rock dam where the seepage barrier is located on the upstream face of the dam body, making inclined contact with the dam shell fill material. It is used to intercept reservoir water seepage and reduce seepage pressure within the dam body. Currently, due to the thin-walled structure and texture characteristics of sloping-wall dams, there is an urgent need for high-frequency, high-precision, and interference-resistant 3D realistic model construction during the construction period, which is a key link in achieving intelligent construction and digital handover.

[0003] In the construction of a 3D reality model of an inclined wall dam, the core wall is made of wide-graded clay compacted by rolling, while the downstream side is adjacent to a rockfill body with significantly different block sizes. The two materials exhibit allometric characteristics in terms of texture, spectrum, and geometric scale: the clay has a fine texture but slow elevation changes, while the rockfill has a coarse texture but dramatic local undulations. Therefore, traditional UAV oblique photography is prone to producing instantaneous shadows under varying sunlight angles. These shadows, falling at the interface between the clay and rockfill, cause image matching errors, resulting in point cloud voids, and misidentifying shadow edges as structural edges. Furthermore, when current image point clouds are fused with the BIM model, the generated mirrored twin mesh model exhibits misaligned boundaries at the interface, which also easily leads to distortion in the generated 3D reality model. Summary of the Invention

[0004] This invention provides a method and system for constructing a three-dimensional real-scene model of a sloping wall dam construction site, which eliminates the risk of shadow edges being misjudged as structural edges from the source, and realizes that the mirror twin mesh model automatically fits together at the interface of different velocities, so that the misalignment of clay and rockfill boundaries is consistent in direction and converges in magnitude, thereby constructing a three-dimensional real-scene model of the sloping wall dam with unified boundaries.

[0005] To solve the above problems, the present invention adopts the following technical solution:

[0006] This invention provides a method for constructing a three-dimensional real-scene model of a sloping wall dam construction site, comprising:

[0007] The image of the inclined wall dam at the construction site is obtained by a drone. A pre-trained image decoupling convolutional neural network is used to separate the instantaneous shadow layer and the steady-state albedo layer in the image, and output a shadowless image with the instantaneous shadow layer removed.

[0008] The initial allometric features of the shadowless image are extracted using a preset feature extraction network. Global pooling and fully connected operations are performed on the initial allometric features to output a probability map with the same pixels as the shadowless image. The feature extraction network includes pointwise convolution and dilated convolution with different dilation rates to capture multi-scale information at once.

[0009] The clay region of the probability map is given an expansion rate less than a preset value, and the rocky region of the probability map is given an expansion rate greater than a preset value. The pixel regions in the probability map that fall into the same dilated convolution are adjusted according to the pixel weighting method. The multi-scale features after pixel weighting are spliced ​​and convolved with the point-by-point convolution features to generate an allometric confidence map. The allometric confidence map is written into the point cloud attribute field to form an initial point cloud with misaligned weights. The allometric confidence map is a single-channel probability map output based on the texture difference between the clay region and the rocky region.

[0010] Using the sloping wall filling boundary measured by the ground GNSS system as the constraint boundary, the initial point cloud is optimized in the hidden space to generate a target point cloud with a pre-offset. The BIM model is coupled with the target point cloud to form a triangular mesh. A pre-trained graph convolutional network is used to transmit the misalignment along the mesh edge of the triangular mesh. The misalignment of each triangular facet in the triangular mesh is encoded as an edge weight and aggregated to generate a misalignment field. A mirror twin mesh model is generated and a three-dimensional real-world model of the sloping wall dam is constructed using the mirror twin mesh model.

[0011] Preferably, the step of using the inclined wall filling boundary measured by the ground GNSS system as the constraint boundary, and optimizing the initial point cloud in the hidden space to generate a target point cloud containing a pre-offset includes:

[0012] The initial point cloud and the sloping wall filling boundary measured by the GNSS system are mapped to the same high-dimensional hidden space through a point-by-point coding network with shared weights, resulting in point cloud hidden vectors and boundary hidden vectors.

[0013] The offset field is output using the point cloud hidden vector as input.

[0014] The offset field is superimposed on the initial point cloud to form a candidate point cloud;

[0015] Determine whether the boundary hidden vector and the hidden vector corresponding to the candidate point cloud come from the same data source;

[0016] If so, push the point cloud boundary towards the sloping wall filling boundary measured by the GNSS system within the hidden space, calculate the pre-offset, and then superimpose the pre-offset onto the candidate point cloud to form the target point cloud.

[0017] Preferably, the transient shadow layer refers to the set of dark pixels generated on the image at a single aerial photography moment due to the combined effects of solar azimuth, terrain undulation and temporary obstruction. The transient shadow layer exists only at the moment of the current exposure and changes immediately as the sun moves or the obstruction is removed, and is unrelated to the inherent reflective properties of the ground surface.

[0018] The steady-state albedo layer refers to the inherent reflectivity image layer exhibited by the Earth's surface under conditions of no shadows and no abrupt changes in the direction of light. The pixel values ​​of the steady-state albedo layer are determined solely by the material's own reflective properties and are unrelated to instantaneous solar angles or temporary shading.

[0019] Preferably, the initial allotropic feature is a feature map obtained by first jointly encoding the different responses of the clay region and the rock pile region in the shadowless image due to differences in texture density, particle size and random surface reflection, including the spatial texture and local context information of the clay region and the rock pile region.

[0020] Furthermore, before separating the instantaneous shadow layer and the steady-state albedo layer in the image using a pre-trained image decoupling convolutional neural network, the method further includes:

[0021] A training set of historical images is obtained, and a shadow branch and an albedo branch are set up for the pre-constructed initial convolutional neural network. The shadow branch adopts a depth-separable convolution superimposed channel attention mechanism to capture information on brightness abrupt changes, edge softening and color temperature shift. The albedo branch adopts convolution combination to learn the reflection properties of shadowless ground surfaces. Each branch outputs a single-channel mask.

[0022] Image pairs from the same region, the same solar azimuth angle, and different time points are selected. Pixels in the image pairs with pixel differences greater than a threshold are marked as instantaneous shadow pixels, and the remaining pixels are marked as steady-state albedo pixels, thus forming supervisory labels.

[0023] The initial convolutional neural network is trained using the historical image training set and supervision labels, and the training phase of the initial convolutional neural network is constrained by a preset loss function. The edge gradient is calculated on the single-channel mask corresponding to the shadow branch, and the edge gradient is constrained to be consistent with the input image gradient to ensure that the shadow spatial phase is preserved.

[0024] When the iteration optimization reaches the loss convergence, the output is the convolution kernel weights with the ability to preserve the phase in the shadow space;

[0025] The weights of the initial convolutional neural network are adjusted according to the convolutional kernel weights to generate an image decoupled convolutional neural network.

[0026] Furthermore, after the output has removed the shadowless image of the transient shadow layer, it also includes:

[0027] Acquire historical images of the same operating conditions and solar azimuth angle from the previous day;

[0028] The weights of the image decoupled convolutional neural network are updated online based on the historical images.

[0029] Furthermore, before transmitting the misalignment along the grid edges of the triangular mesh using a pre-trained graph convolutional network, the method further includes:

[0030] Obtain the training sample dataset;

[0031] The initial graph convolutional network is trained in a supervised manner using the training sample dataset, and multiple sets of parameter combinations are output.

[0032] The optimal parameter combination is determined from the multiple parameter combinations, and the initial graph convolutional network is configured according to the optimal parameter combination to obtain the trained graph convolutional network.

[0033] Preferably, the step of constructing a 3D reality model of the inclined wall dam using the mirrored twin mesh model includes:

[0034] The shadowless image is superimposed onto the mirror twin mesh model according to the projection relationship between the image and the mesh model, and the texture coordinates are driven to move so that the texture and geometry misalignment correspond on the mesh surface to generate a three-dimensional real scene model.

[0035] Preferably, the step of performing global pooling and fully connected operations on the initial allometric features to output a probability map with the same pixels as the shadowless image includes:

[0036] The texture preservation path and geometry preservation path are simultaneously activated for the initial allotropic features. The descriptors obtained from the two preservation paths are spliced ​​and compressed to form a global descriptor. The texture preservation path retains the target response of the rock pile region in a channel selection manner, and the geometry preservation path retains the slowly varying components of the clay region in a low-frequency truncation manner.

[0037] The global descriptor is fitted to the original spatial resolution of the shadowless image and multiplied pixel-by-pixel with the initial allometric feature to obtain the resonance feature;

[0038] Perform a bilinear fully connected operation on the resonance features to output an initial probability map;

[0039] An optimal transport model is constructed within the space of the initial probability map, with joint statistics of texture and geometry as the cost matrix. The confidence between the clay region and the riprap region of the initial probability map is adjusted to an optimal balance state using the optimal transport model. The adjusted initial probability map is then smoothed to obtain a probability map with the same number of pixels as the shadowless image.

[0040] This invention also provides a three-dimensional reality model construction system for inclined wall dam construction sites, comprising:

[0041] The separation module is used to acquire images of the inclined wall dam taken by the UAV at the construction site, and to use a pre-trained image decoupling convolutional neural network to separate the instantaneous shadow layer and steady-state albedo layer in the image, and output a shadowless image with the instantaneous shadow layer removed.

[0042] The extraction module is used to extract the initial allometric features of the shadowless image using a preset feature extraction network, perform global pooling and fully connected operations on the initial allometric features, and output a probability map with the same pixels as the shadowless image. The feature extraction network includes pointwise convolution and dilated convolution with different dilation rates, which is used to capture multi-scale information at one time.

[0043] An adjustment module is used to assign an expansion rate less than a preset value to the clay region of the probability map, and an expansion rate greater than a preset value to the boulders region of the probability map. It also adjusts pixel regions in the probability map that fall within the same dilated convolution according to a pixel weighting method, and concatenates and fuses the pixel-weighted multi-scale features with point-by-point convolutional features to generate an allometric confidence map. The allometric confidence map is written into the point cloud attribute field to form an initial point cloud with misaligned weights. The allometric confidence map is a single-channel probability map output based on the texture differences between the clay region and the boulders region.

[0044] The construction module is used to optimize the initial point cloud in the hidden space with the inclined wall filling boundary measured by the ground GNSS system as the constraint boundary, generate a target point cloud with pre-offset, couple the BIM model with the target point cloud into a triangular mesh, use a pre-trained graph convolutional network to transmit the misalignment amount along the mesh edge of the triangular mesh, encode the misalignment amount of each triangular facet in the triangular mesh as edge weight, aggregate to generate a misalignment field, generate a mirror twin mesh model, and use the mirror twin mesh model to construct a three-dimensional real scene model of the inclined wall dam.

[0045] Compared with the prior art, the technical solution of the present invention has at least the following advantages:

[0046] This invention provides a method and system for constructing a 3D real-scene model of a sloping wall dam construction site. It utilizes an image-decoupled convolutional neural network to separate the instantaneous shadow layer from the steady-state albedo layer in the original image, outputting a shadow-free image. This eliminates the risk of shadow edges being misjudged as structural edges from the source. Subsequently, a small expansion rate is assigned to the clay region of the probability map, and a large expansion rate is assigned to the rockfill region. A pixel-weighted method is then used to redistribute weights to pixels with different speeds within the same dilated convolution, ensuring that fine textures are not smoothed by large expansion rates and large-scale blocks are not fragmented by small expansion rates.

[0047] Meanwhile, after concatenating and fusion the pixel-weighted multi-scale features with the point-by-point convolutional features, an allometric confidence map is generated. This confidence map is directly written into the point cloud attribute field to form an initial point cloud with misaligned weights, realizing a one-to-one mapping between allometric features and point cloud attributes. This allows for the quantification of texture differences between clay and riprap regions through the allometric confidence map, and adaptive adjustment of the receptive field during the same forward propagation to avoid over-smoothing of allometric regions.

[0048] Furthermore, the allometric confidence graph embeds point cloud attribute fields in the form of misaligned weights, which also ensures that the misalignment received by subsequent graph convolutional networks within the same triangular facet is in the same direction, thus reducing the number of convergence iterations for the mirror twin mesh model.

[0049] Furthermore, using the measured boundary of the sloping wall as a hard constraint, the initial point cloud is optimized in the hidden space to generate a target point cloud containing the target offset. After being coupled with the BIM model into a triangular mesh, the trained graph convolutional network transmits the misalignment along the mesh edge, encodes the misalignment of each triangular facet as an edge weight, and aggregates to generate a continuous misalignment field. Finally, the mirror twin mesh model is automatically fitted at the interface of different velocities, so that the misalignment of the clay and rockfill boundaries is consistent in direction and converges in magnitude, thereby further solving the problem of misaligned boundaries appearing at the interface of the model. Thus, a three-dimensional real-scene model of the sloping wall dam with unified boundaries is constructed, improving the accuracy and realism of the sloping wall dam real-scene model. Attached Figure Description

[0050] Figure 1 This is a flowchart of an embodiment of a method for constructing a three-dimensional real-scene model of a sloping wall dam construction site according to the present invention;

[0051] Figure 2 This is a flowchart illustrating another embodiment of the method for constructing a three-dimensional real-scene model of a sloping wall dam construction site according to the present invention.

[0052] Figure 3 This is a structural block diagram of an embodiment of a three-dimensional real-scene model construction system for a sloping wall dam construction site according to the present invention. Detailed Implementation

[0053] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0054] Please refer to Figure 1 This invention provides a method for constructing a three-dimensional real-scene model of a sloping wall dam construction site, comprising the following steps:

[0055] S11. Acquire images of the inclined wall dam at the construction site taken by the UAV, and use a pre-trained image decoupling convolutional neural network to separate the instantaneous shadow layer and steady-state albedo layer in the image, and output a shadowless image with the instantaneous shadow layer removed.

[0056] S12. Extract the initial allometric features of the shadowless image using a preset feature extraction network, perform global pooling and fully connected operations on the initial allometric features, and output a probability map with the same pixels as the shadowless image. The feature extraction network includes pointwise convolution and dilated convolution with different dilation rates, which are used to capture multi-scale information at one time.

[0057] S13. Assign an expansion rate less than a preset value to the clay region of the probability map, and an expansion rate greater than a preset value to the boulders region of the probability map. Adjust the pixel regions in the probability map that fall into the same dilated convolution according to the pixel weighting method. Then, stitch and convolve the multi-scale features after pixel weighting with the point-by-point convolution features to generate an allometric confidence map. Write the allometric confidence map into the point cloud attribute field to form an initial point cloud with misaligned weights. The allometric confidence map is a single-channel probability map output based on the texture difference between the clay region and the boulders region.

[0058] S14. Using the sloping wall filling boundary measured by the ground GNSS system as the constraint boundary, optimize the initial point cloud in the hidden space to generate a target point cloud with a pre-offset. Couple the BIM model with the target point cloud into a triangular mesh. Use a pre-trained graph convolutional network to transmit the misalignment along the mesh edge of the triangular mesh. Encode the misalignment of each triangular facet in the triangular mesh as an edge weight, aggregate to generate a misalignment field, generate a mirror twin mesh model, and use the mirror twin mesh model to construct a three-dimensional real-scene model of the sloping wall dam.

[0059] This embodiment utilizes drones to capture images of the inclined wall dam construction site. Due to complex lighting conditions at the site, transient shadows often exist in the images, such as dynamic shadows caused by changes in the angle of sunlight, equipment, and personnel obstruction. A pre-trained image decoupling convolutional neural network is used to decompose the image into two independent layers: a transient shadow layer and a steady-state albedo layer. The transient shadow layer refers to the set of dark pixels generated on the image at a single aerial photographic moment due to the combined effects of the sun's azimuth angle, terrain undulations, and temporary obstructions. This transient shadow layer exists only at the moment of exposure and changes immediately with the movement of the sun or the removal of the obstruction, independent of the inherent reflectivity of the ground surface. The steady-state albedo layer refers to the image layer showing the inherent reflectivity of the ground surface under conditions of no shadows and no abrupt changes in the direction of light. The pixel values ​​of the steady-state albedo layer are determined solely by the reflectivity of the material itself and are independent of the transient sun angle or temporary obstruction.

[0060] The final output is a shadowless image with shadows removed, eliminating shadow interference. By using an image decoupling convolutional neural network, the transient shadow layer and the steady-state albedo layer in the original image are separated, and a shadowless image is output, thereby eliminating the risk of shadow edges being misjudged as structural edges from the source.

[0061] Initial allometric features are extracted from the shadowless image. The initial allometric features are feature maps obtained by first jointly encoding the different responses of the clay region and the rock pile region in the shadowless image due to differences in texture density, particle size and random surface reflection. The initial allometric features include the spatial texture and local context information of the clay region and the rock pile region.

[0062] The feature extraction network can employ pointwise convolution to capture local details (such as the smooth texture of clay surfaces) and dilated convolution to expand the receptive field and capture multi-scale information at once (such as large stones and small gaps in a rock pile area).

[0063] By using global pooling and fully connected operations, a probability map is generated that corresponds one-to-one with the pixels of the shadowless image. Each pixel value represents the initial confidence that the location belongs to a certain type of material (such as clay or rubble).

[0064] Based on the probability map, the clay region and the boulders region are distinguished. Due to the difference in texture complexity between the two, the texture of the clay region is uniform and a smaller dilation rate is required to avoid over-smoothing. The texture of the boulders region is complex (such as the edges of the boulders and pores), so a larger dilation rate is required to capture the global structure. The weight of pixels within the same dilated convolution window is adjusted by the pixel weighting method (such as adjusting the weight of edge pixels in the boulders region to be higher), so that fine textures are not smoothed by a large dilation rate and large-scale blocks are not fragmented by a small dilation rate.

[0065] Subsequently, the weighted multi-scale features are concatenated and fused with the pointwise convolutional features to generate an allometric confidence map. This allometric confidence map is a single-channel probability map; a higher value indicates a higher confidence level in material classification at that location. This map is then written into the point cloud attribute field to form an initial point cloud with misaligned weights, where each point is associated with a confidence weight. This achieves a one-to-one mapping between allometric features and point cloud attributes, quantifying the texture differences between clay and riprap regions through the allometric confidence map. The receptive field is adaptively adjusted during the same forward propagation to prevent over-smoothing of allometric regions. Furthermore, embedding the allometric confidence map into the point cloud attribute field with misaligned weights ensures that the misalignment received by subsequent graph convolutional networks within the same triangular facet is oriented in the same direction, reducing the convergence iterations of the mirror twin mesh model.

[0066] Using the sloping wall filling boundary measured by ground GNSS as a hard constraint, the initial point cloud is optimized in the hidden space to correct the boundary offset caused by material classification errors, generating a target point cloud with pre-offset, i.e., compensating for possible misalignment in advance. The BIM model is aligned with the target point cloud to generate a triangular mesh. A pre-trained graph convolutional network is used to propagate the misalignment along the mesh edges (e.g., a triangular facet needs to be offset upwards due to material differences). The misalignment is encoded as edge weights (e.g., a larger weight indicates that the two facets of that edge need to be adjusted more). All edge weights are aggregated to generate a misalignment field, i.e., the displacement vector distribution of the entire mesh. Based on the mirror twin mesh model generated by the misalignment field and symmetrical to the actual construction state, i.e., the theoretical misalignment-free model, the final three-dimensional real scene model is constructed by comparing the differences between the actual mesh and the twin mesh to reflect the actual construction deviation. This ultimately enables the mirror twin mesh model to automatically fit together at the interface of different velocities, making the misalignment of the clay and rockfill boundaries consistent in direction and convergent in magnitude. This further solves the problem of misaligned boundaries appearing at the interface, thus constructing a three-dimensional real scene model of the inclined wall dam with unified boundaries, improving the accuracy and realism of the inclined wall dam real scene model.

[0067] In one embodiment, please refer to Figure 2 The step of using the sloping wall filling boundary measured by the ground GNSS system as the constraint boundary, and optimizing the initial point cloud in the hidden space to generate a target point cloud containing a pre-offset includes:

[0068] S141. The initial point cloud and the sloping wall filling boundary measured by the GNSS system are mapped to the same high-dimensional hidden space through a point-by-point coding network with shared weights, to obtain the point cloud hidden vector and the boundary hidden vector.

[0069] S142. Using the point cloud hidden vector as input, output the offset field;

[0070] S143. The offset field is superimposed on the initial point cloud to form a candidate point cloud;

[0071] S144. Determine whether the boundary hidden vector and the hidden vector corresponding to the candidate point cloud come from the same data source;

[0072] S145. If so, push the point cloud boundary towards the sloping wall filling boundary measured by the GNSS system within the hidden space, calculate the pre-offset, and superimpose the pre-offset onto the candidate point cloud to form the target point cloud.

[0073] In this embodiment, each point in the initial point cloud already carries a misalignment weight. The boundary of the inclined wall filling measured by the GNSS system is a set of real-world coordinates. The two types of data are sent into a point-by-point coding network with shared weights: the same set of network parameters and the same set of coordinate normalization ensure that they are projected into the same high-dimensional hidden space, and two types of hidden vectors are output: one type represents the point cloud and the other type represents the boundary. The two can be directly compared in the same vector space to eliminate differences in dimensions, density, sampling unevenness, etc., which facilitates subsequent comparison.

[0074] Using the hidden vectors of the point cloud as the sole input, the network outputs an offset field of the same length as the number of points. Each offset in the offset field depends only on the local context of that point, providing preliminary suggestions for the direction and distance of movement of that point. The resulting offset field is directly added to the coordinates of the initial point cloud to obtain a candidate point cloud. At this point, the candidate point cloud is closer to the true boundary than the initial point cloud, forming an evaluable and iterative intermediate state, but it has not yet been subjected to the hard constraints of actual measurements from the GNSS system.

[0075] The candidate point clouds are then fed into the same point-by-point coding network to obtain their corresponding hidden vectors. The system determines whether these hidden vectors and the boundary hidden vectors measured by the GNSS system originate from the same data distribution, that is, whether the two overlap in the hidden space to the point that they can be considered to have the same origin. The determination method can be distance, angle, or distribution statistics.

[0076] Once the source is determined, the boundary region of the candidate point cloud is advanced towards the measured boundary of the GNSS system in the hidden space. The advancement amount is recorded as a pre-offset, which is then superimposed back onto the candidate point cloud to form the final target point cloud. This target point cloud can simultaneously satisfy the requirements of internally retained material classification confidence, strict alignment between the boundary and the measured GNSS system data, and all adjustments are completed in the hidden space, avoiding discontinuities or abrupt changes in the original coordinate system. This allows for the accurate embedding of real-world hard measurements into the point cloud, ensuring that subsequent triangular mesh, dislocation field, and mirror twin mesh models operate on the corrected geometry.

[0077] This embodiment can complete the hard alignment of boundaries at the point cloud stage. When coupling the BIM model with the point cloud and propagating misalignment through graph convolution, only local fine-tuning and quality assessment need to be focused on, reducing the number of iterations. At the same time, it avoids unreasonable dragging of internal points for forced alignment, thus maintaining the physical authenticity of the material distribution and geometric shape inside the dam body.

[0078] In one embodiment, before separating the transient shadow layer and the steady-state albedo layer in the image using a pre-trained image decoupling convolutional neural network, the method further includes:

[0079] A training set of historical images is obtained, and a shadow branch and an albedo branch are set up for the pre-constructed initial convolutional neural network. The shadow branch adopts a depth-separable convolution superimposed channel attention mechanism to capture information on brightness abrupt changes, edge softening and color temperature shift. The albedo branch adopts convolution combination to learn the reflection properties of shadowless ground surfaces. Each branch outputs a single-channel mask.

[0080] Image pairs from the same region, the same solar azimuth angle, and different time points are selected. Pixels in the image pairs with pixel differences greater than a threshold are marked as instantaneous shadow pixels, and the remaining pixels are marked as steady-state albedo pixels, thus forming supervisory labels.

[0081] The initial convolutional neural network is trained using the historical image training set and supervision labels, and the training phase of the initial convolutional neural network is constrained by a preset loss function. The edge gradient is calculated on the single-channel mask corresponding to the shadow branch, and the edge gradient is constrained to be consistent with the input image gradient to ensure that the shadow spatial phase is preserved.

[0082] When the iteration optimization reaches the loss convergence, the output is the convolution kernel weights with the ability to preserve the phase in the shadow space;

[0083] The weights of the initial convolutional neural network are adjusted according to the convolutional kernel weights to generate an image decoupled convolutional neural network.

[0084] Before deploying the network to the project site, a dedicated shadow decoupling model needs to be trained in an offline environment. This pre-processing involves multiple steps, each completed in a single phase in the laboratory, without subsequently impacting the construction schedule.

[0085] First, an initial convolutional neural network is constructed and divided into a two-branch structure: a shadow branch and an albedo branch. The shadow branch employs a depthwise separable convolutional stacked channel attention mechanism to focus on shadow features such as brightness abrupt changes, edge softening, and color temperature shifts; the albedo branch learns the inherent reflectivity properties of shadowless surfaces through convolutional combination. Each branch outputs a single-channel mask to identify the shadow region and the steady-state albedo region, respectively. Image pairs from the same region, with the same solar azimuth angle, but at different time phases are selected, and instantaneous shadow pixels and steady-state albedo pixels are automatically classified based on pixel difference thresholds to form pixel-level supervised labels, eliminating the need for manual annotation throughout the process.

[0086] Then, the network is iteratively optimized using the historical image training set and the aforementioned labels. Based on the cross loss function, the edge gradient can be calculated on the single-channel mask output by the shadow branch, and the gradient is constrained to be consistent with the gradient of the input image to ensure that the spatial phase information of the shadow is preserved.

[0087] Finally, when the loss function converges, the output convolutional kernel weights with shadow space phase preservation capability are used to assign the obtained weights to the initial network to generate an image decoupled convolutional neural network, which is used for the separation of instantaneous shadows and steady-state albedo in subsequent field images.

[0088] This embodiment introduces edge gradient consistency constraints, enabling the shadow mask to retain complete spatial phase while achieving accurate separation, effectively preventing shadow edges from being misidentified as material boundaries. The resulting shadow-free image significantly improves the accuracy of subsequent feature extraction, material classification, and geometric reconstruction.

[0089] In one embodiment, after the output has removed the shadowless image of the transient shadow layer, it further includes:

[0090] Acquire historical images of the same operating conditions and solar azimuth angle from the previous day;

[0091] The weights of the image decoupled convolutional neural network are updated online based on the historical images.

[0092] This embodiment, after outputting a shadowless image with transient shadow layers removed, can add an online update step to ensure that the image decoupling convolutional neural network maintains optimal separation performance as construction progresses and lighting conditions change. Specifically, after the day's work is completed, the system automatically retrieves historical images from the previous day under the same conditions and at the same solar azimuth angle. These images have undergone shadow decoupling processing and possess geometric and lighting references consistent with the current image, serving as a reliable reference.

[0093] Subsequently, using historical images from the previous day as a benchmark, the weights of the decoupled convolutional neural network for the current image are fine-tuned. Specifically, the shadowless results of the historical images are considered pseudo-true values. The difference between the current network output and the pseudo-true values ​​is calculated, and the convolutional kernel parameters are updated through a gradient backpropagation mechanism. The learning rate is set to a small value to ensure that the weights adapt to the latest data distribution while preserving existing knowledge.

[0094] This embodiment enables the network to continuously track surface changes and light drift at the construction site, avoiding data distribution shifts caused by seasons, dust, or filling progress. The online-updated network can provide more accurate shadow separation results in the following day's work, thus ensuring the temporal consistency of shadowless image sequences.

[0095] In one embodiment, before transmitting the misalignment along the grid edges of the triangular mesh using a pre-trained graph convolutional network, the method further includes:

[0096] Obtain the training sample dataset;

[0097] The initial graph convolutional network is trained in a supervised manner using the training sample dataset, and multiple sets of parameter combinations are output.

[0098] The optimal parameter combination is determined from the multiple parameter combinations, and the initial graph convolutional network is configured according to the optimal parameter combination to obtain the trained graph convolutional network.

[0099] This embodiment can collect triangular mesh samples covering different dam sections, different construction stages, and different material distributions. Each sample is accompanied by a known true value of the misalignment. The true value is obtained through high-precision measurement methods, such as combined observation with total station and laser scanning, and mapped to the grid edges to form a training sample dataset, ensuring the authority and representativeness of the training data.

[0100] Supervised training of the initial graph convolutional network is performed using a training sample dataset. The network takes the geometric features, adjacency relationships, and material properties of the mesh edges as input and outputs the corresponding misalignment. During training, multiple sets of hyperparameter configurations are used, each of which may include the number of layers, the number of channels, the type of aggregation function, etc., and are optimized in parallel iteratively. Finally, multiple sets of parameter combinations and their validation errors are output.

[0101] The accuracy and stability of each parameter set were evaluated on the validation set. The set with the smallest error and a stable convergence curve was selected as the optimal parameter combination. This combination was then written into the initial graph convolutional network (GCNN) to solidify the weights, resulting in a trained GCNN. Through rigorous supervised training and parameter optimization, the GCNN was able to accurately perceive local geometric changes in the mesh and reasonably propagate the dislocation. When applied to a real triangular mesh, the network can smoothly and efficiently propagate the dislocation along the mesh edges to the entire dam surface while maintaining topological consistency. The resulting dislocation field not only fits the measured boundary but also preserves the details of the material partitioning, providing a highly reliable geometric correction basis for the construction of a mirror twin mesh model.

[0102] In one embodiment, constructing a 3D reality model of the inclined wall dam using the mirrored twin mesh model includes:

[0103] The shadowless image is superimposed onto the mirror twin mesh model according to the projection relationship between the image and the mesh model, and the texture coordinates are driven to move so that the texture and geometry misalignment correspond on the mesh surface to generate a three-dimensional real scene model.

[0104] Based on the internal and external orientation elements during UAV imaging, the precise projection matrix between the shadowless image and the mirror twin mesh model is calculated to ensure that each image pixel has a unique ray correspondence in three-dimensional space. The shadowless image is projected onto the surface of the mirror twin mesh frame by frame, and the occluded area is removed using the Z-buffer hidden surface removal algorithm, while retaining the texture information of the visible part to form the initial texture map.

[0105] Based on the patch-level displacement vector provided by the misalignment field, the texture coordinates are translated and rotated so that the texture blocks that were originally offset from the geometry are accurately placed into the corresponding mesh patches, thus achieving a one-to-one correspondence between texture and geometric misalignment.

[0106] After completing the texture coordinate relocation, multi-band fusion is performed on the texture seams of adjacent images to eliminate the difference in light intensity. Finally, a 3D real-world model of the sloping wall dam with realistic colors and centimeter-level geometric accuracy is output. By introducing a texture coordinate refinement mechanism driven by the misalignment field, the model can effectively eliminate ghosting, stretching and seam misalignment caused by the incomplete alignment between the construction period images and the final mesh while maintaining high geometric fidelity.

[0107] In one embodiment, performing global pooling and fully connected operations on the initial allometric features to output a probability map with the same pixels as the shadowless image includes:

[0108] The texture preservation path and geometry preservation path are simultaneously activated for the initial allotropic features. The descriptors obtained from the two preservation paths are spliced ​​and compressed to form a global descriptor. The texture preservation path retains the target response of the rock pile region in a channel selection manner, and the geometry preservation path retains the slowly varying components of the clay region in a low-frequency truncation manner.

[0109] The global descriptor is fitted to the original spatial resolution of the shadowless image and multiplied pixel-by-pixel with the initial allometric feature to obtain the resonance feature;

[0110] Perform a bilinear fully connected operation on the resonance features to output an initial probability map;

[0111] An optimal transport model is constructed within the space of the initial probability map, with joint statistics of texture and geometry as the cost matrix. The confidence between the clay region and the riprap region of the initial probability map is adjusted to an optimal balance state using the optimal transport model. The adjusted initial probability map is then smoothed to obtain a probability map with the same number of pixels as the shadowless image.

[0112] In this embodiment, the initial allometric features are simultaneously entered into the texture preservation path and the geometry preservation path. The texture preservation path uses a channel selection mechanism to truncate the target response in the rubble area, suppressing redundant details. This target response can be a high-amplitude response within the top 15% of the total amplitude. The geometry preservation path uses low-frequency truncation to retain the gradually varying components of the clay area to reduce noise interference. Here, the gradually varying components refer to signals with low spatial frequencies and gentle changes in the image, corresponding to parts where grayscale or color shows a gradual transition over a large area, rather than abrupt edges.

[0113] The descriptors output from the two paths are spliced ​​and compressed to form a unified global descriptor, which can take into account both the saliency of the riprap texture and the continuity of the clay geometry.

[0114] Subsequently, the global descriptor is expanded at the original spatial resolution of the shadowless image and multiplied pixel by pixel with the initial allometric features to obtain the resonance features, so that the global semantic information corresponds to the local details point by point, thereby enhancing the sensitivity of material discrimination while maintaining the spatial positioning accuracy.

[0115] Bilinear fully connected operations are performed on the resonance features, and quadratic terms are introduced into the feature interaction layer to enhance nonlinear discrimination capabilities, outputting an initial probability map to achieve preliminary pixel-level classification of clay and riprap. Within the space of the initial probability map, a cost matrix is ​​constructed using joint texture and geometric statistics to establish an optimal transport model. By solving the transport plan, the confidence scores of clay and riprap regions are globally optimally redistributed to achieve an optimal balance and eliminate local misclassifications. Subsequently, edge smoothing is applied to obtain a final probability map that corresponds one-to-one with the pixels of the shadowless image.

[0116] In this embodiment, the dual-path mechanism of texture and geometry effectively decouples the high-frequency information of riprap from the low-frequency morphology of clay, avoiding category confusion caused by a single feature; the global and local resonance strategy enhances the receptive field while preserving spatial resolution, preventing edge blurring; optimal post-transmission processing further corrects the confidence distribution, enabling the probability map to exhibit natural and continuous gradient changes in the material transition region. The final output probability map provides a highly reliable and high-fidelity input basis for the differential expansion of clay and riprap and the assignment of misaligned weights, thereby significantly improving the accuracy and stability of the allometric confidence map.

[0117] Please refer to Figure 3 The present invention also provides a three-dimensional real-scene model construction system for inclined wall dam construction sites, comprising:

[0118] The separation module 31 is used to acquire images of the inclined wall dam taken by the UAV at the construction site, and use a pre-trained image decoupling convolutional neural network to separate the instantaneous shadow layer and steady-state albedo layer in the image, and output a shadowless image with the instantaneous shadow layer removed.

[0119] Extraction module 32 is used to extract the initial allometric features of the shadowless image using a preset feature extraction network, perform global pooling and fully connected operations on the initial allometric features, and output a probability map with the same pixels as the shadowless image. The feature extraction network includes pointwise convolution and dilated convolution with different dilation rates, which is used to capture multi-scale information at one time.

[0120] The adjustment module 33 is used to assign an expansion rate less than a preset value to the clay region of the probability map, and an expansion rate greater than a preset value to the rock pile region of the probability map. It also adjusts the pixel regions in the probability map that fall into the same dilated convolution according to the pixel weighting method, and splices and convolves the multi-scale features after pixel weighting with the point-by-point convolution features to generate an allometric confidence map. The allometric confidence map is written into the point cloud attribute field to form an initial point cloud with misaligned weights. The allometric confidence map is a single-channel probability map output based on the texture difference between the clay region and the rock pile region.

[0121] The construction module 34 is used to optimize the initial point cloud in the hidden space with the inclined wall filling boundary measured by the ground GNSS system as the constraint boundary, generate a target point cloud with pre-offset, couple the BIM model with the target point cloud into a triangular mesh, use a pre-trained graph convolutional network to transmit the misalignment amount along the mesh edge of the triangular mesh, encode the misalignment amount of each triangular facet in the triangular mesh as edge weight, aggregate to generate a misalignment field, generate a mirror twin mesh model, and use the mirror twin mesh model to construct a three-dimensional real scene model of the inclined wall dam.

[0122] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0123] In one embodiment, the present invention also proposes a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned method for constructing a three-dimensional reality model of a sloping wall dam construction site. The storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0124] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0125] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for constructing a three-dimensional real-scene model of a sloping wall dam construction site, characterized in that, include: The image of the inclined wall dam at the construction site is obtained by a drone. A pre-trained image decoupling convolutional neural network is used to separate the instantaneous shadow layer and the steady-state albedo layer in the image, and output a shadowless image with the instantaneous shadow layer removed. The initial allometric features of the shadowless image are extracted using a preset feature extraction network. Global pooling and fully connected operations are performed on the initial allometric features to output a probability map with the same pixels as the shadowless image. The feature extraction network includes pointwise convolution and dilated convolution with different dilation rates to capture multi-scale information at once. The clay region of the probability map is given an expansion rate less than a preset value, and the rocky region of the probability map is given an expansion rate greater than a preset value. The pixel regions in the probability map that fall into the same dilated convolution are adjusted according to the pixel weighting method. The multi-scale features after pixel weighting are spliced ​​and convolved with the point-by-point convolution features to generate an allometric confidence map. The allometric confidence map is written into the point cloud attribute field to form an initial point cloud with misaligned weights. The allometric confidence map is a single-channel probability map output based on the texture difference between the clay region and the rocky region. Using the sloping wall filling boundary measured by the ground GNSS system as the constraint boundary, the initial point cloud is optimized in the hidden space to generate a target point cloud with a pre-offset. The BIM model is coupled with the target point cloud to form a triangular mesh. A pre-trained graph convolutional network is used to transmit the misalignment along the mesh edge of the triangular mesh. The misalignment of each triangular facet in the triangular mesh is encoded as an edge weight and aggregated to generate a misalignment field. A mirror twin mesh model is generated and a three-dimensional real-world model of the sloping wall dam is constructed using the mirror twin mesh model.

2. The method according to claim 1, characterized in that, The process of using the sloping wall filling boundary measured by the ground GNSS system as the constraint boundary, and optimizing the initial point cloud in the hidden space to generate a target point cloud with a pre-offset includes: The initial point cloud and the sloping wall filling boundary measured by the GNSS system are mapped to the same high-dimensional hidden space through a point-by-point coding network with shared weights, resulting in point cloud hidden vectors and boundary hidden vectors. Using the point cloud hidden vector as input, the output is an offset field; The offset field is superimposed on the initial point cloud to form a candidate point cloud; Determine whether the boundary hidden vector and the hidden vector corresponding to the candidate point cloud come from the same data source; If so, push the point cloud boundary towards the sloping wall filling boundary measured by the GNSS system within the hidden space, calculate the pre-offset, and then superimpose the pre-offset onto the candidate point cloud to form the target point cloud.

3. The method according to claim 1, characterized in that, The transient shadow layer refers to the set of dark pixels generated on the image at a single aerial photography moment due to the combined effects of the sun's azimuth angle, terrain undulations, and temporary obstruction. The transient shadow layer exists only at the moment of the current exposure and changes immediately as the sun moves or the obstruction is removed, and is unrelated to the inherent reflective properties of the earth's surface. The steady-state albedo layer refers to the inherent reflectivity image layer exhibited by the Earth's surface under conditions of no shadows and no abrupt changes in the direction of light. The pixel values ​​of the steady-state albedo layer are determined solely by the material's own reflective properties and are unrelated to instantaneous solar angles or temporary shading.

4. The method according to claim 1, characterized in that, The initial allotropic feature is a feature map obtained by first jointly encoding the different responses of the clay region and the rock pile region in the shadowless image due to differences in texture density, particle size and random surface reflection. It includes the spatial texture and local context information of the clay region and the rock pile region.

5. The method according to claim 1, characterized in that, Before separating the instantaneous shadow layer and the steady-state albedo layer in the image using a pre-trained image decoupling convolutional neural network, the method further includes: A training set of historical images is obtained, and a shadow branch and an albedo branch are set up for the pre-constructed initial convolutional neural network. The shadow branch adopts a depth-separable convolution superimposed channel attention mechanism to capture information on brightness abrupt changes, edge softening and color temperature shift. The albedo branch adopts convolution combination to learn the reflection properties of shadowless ground surfaces. Each branch outputs a single-channel mask. Image pairs from the same region, the same solar azimuth angle, and different time points are selected. Pixels in the image pairs with pixel differences greater than a threshold are marked as instantaneous shadow pixels, and the remaining pixels are marked as steady-state albedo pixels, thus forming supervisory labels. The initial convolutional neural network is trained using the historical image training set and supervision labels, and the training phase of the initial convolutional neural network is constrained by a preset loss function. The edge gradient is calculated on the single-channel mask corresponding to the shadow branch, and the edge gradient is constrained to be consistent with the input image gradient to ensure that the shadow spatial phase is preserved. When the iteration optimization reaches the loss convergence, the output is the convolution kernel weights with the ability to preserve the phase in the shadow space; The weights of the initial convolutional neural network are adjusted according to the convolutional kernel weights to generate an image decoupled convolutional neural network.

6. The method according to claim 1, characterized in that, The output, after removing the shadowless image of the transient shadow layer, also includes: Acquire historical images of the same operating conditions and solar azimuth angle from the previous day; The weights of the image decoupled convolutional neural network are updated online based on the historical images.

7. The method according to claim 1, characterized in that, Before transmitting the misalignment along the edges of the triangular mesh using a pre-trained graph convolutional network, the method further includes: Obtain the training sample dataset; The initial graph convolutional network is trained in a supervised manner using the training sample dataset, and multiple sets of parameter combinations are output. The optimal parameter combination is determined from the multiple parameter combinations, and the initial graph convolutional network is configured according to the optimal parameter combination to obtain the trained graph convolutional network.

8. The method according to claim 1, characterized in that, The construction of a 3D reality model of the inclined wall dam using the mirrored twin mesh model includes: The shadowless image is superimposed onto the mirror twin mesh model according to the projection relationship between the image and the mesh model, and the texture coordinates are driven to move so that the texture and geometry misalignment correspond on the mesh surface to generate a three-dimensional real scene model.

9. The method according to claim 1, characterized in that, The step of performing global pooling and fully connected operations on the initial allometric features to output a probability map with the same pixels as the shadowless image includes: The texture preservation path and geometry preservation path are simultaneously activated for the initial allotropic features. The descriptors obtained from the two preservation paths are spliced ​​and compressed to form a global descriptor. The texture preservation path retains the target response of the rock pile region in a channel selection manner, and the geometry preservation path retains the slowly varying components of the clay region in a low-frequency truncation manner. The global descriptor is fitted to the original spatial resolution of the shadowless image and multiplied pixel-by-pixel with the initial allometric feature to obtain the resonance feature; Perform a bilinear fully connected operation on the resonance features to output an initial probability map; An optimal transport model is constructed within the space of the initial probability map, with joint statistics of texture and geometry as the cost matrix. The confidence between the clay region and the riprap region of the initial probability map is adjusted to an optimal balance state using the optimal transport model. The adjusted initial probability map is then smoothed to obtain a probability map with the same number of pixels as the shadowless image.

10. A three-dimensional real-scene model construction system for a sloping wall dam construction site, characterized in that, include: The separation module is used to acquire images of the inclined wall dam taken by the UAV at the construction site, and to use a pre-trained image decoupling convolutional neural network to separate the instantaneous shadow layer and steady-state albedo layer in the image, and output a shadowless image with the instantaneous shadow layer removed. The extraction module is used to extract the initial allometric features of the shadowless image using a preset feature extraction network, perform global pooling and fully connected operations on the initial allometric features, and output a probability map with the same pixels as the shadowless image. The feature extraction network includes pointwise convolution and dilated convolution with different dilation rates, which is used to capture multi-scale information at one time. An adjustment module is used to assign an expansion rate less than a preset value to the clay region of the probability map, and an expansion rate greater than a preset value to the boulders region of the probability map. It also adjusts pixel regions in the probability map that fall within the same dilated convolution according to a pixel weighting method, and concatenates and fuses the pixel-weighted multi-scale features with point-by-point convolutional features to generate an allometric confidence map. The allometric confidence map is written into the point cloud attribute field to form an initial point cloud with misaligned weights. The allometric confidence map is a single-channel probability map output based on the texture differences between the clay region and the boulders region. The construction module is used to optimize the initial point cloud in the hidden space with the inclined wall filling boundary measured by the ground GNSS system as the constraint boundary, generate a target point cloud with pre-offset, couple the BIM model with the target point cloud into a triangular mesh, use a pre-trained graph convolutional network to transmit the misalignment amount along the mesh edge of the triangular mesh, encode the misalignment amount of each triangular facet in the triangular mesh as edge weight, aggregate to generate a misalignment field, generate a mirror twin mesh model, and use the mirror twin mesh model to construct a three-dimensional real scene model of the inclined wall dam.

Images