Methods, devices, equipment, media, and products for reconstructing 3D models of heavy-haul railways

By training a 3D scene reconstruction model using multi-view images and initial scene metadata, and combining pseudo-true image updates and Jacobian matrix deformation field repair of explicit triangular mesh models, the problems of environmental noise interference and geometric repair in the 3D reconstruction of heavy-haul railways were solved, and the reconstruction of high-fidelity railway facility models was achieved.

CN123118973APending Publication Date: 2026-11-17SHUOHUANG RAILWAY DEV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611523255.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-29
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional 3D reconstruction methods struggle to effectively remove environmental noise interference in the complex environment of heavy-haul railways, affecting texture quality and scene representation, and lack precise geometric repair capabilities.

Method used

By acquiring multi-view images and initial scene metadata, a 3D scene reconstruction model is trained using preset prior constraints. By combining pseudo-true image updates and Jacobian matrix deformation field repair of explicit triangular mesh models, denoising and geometric restoration of the 3D model of heavy-haul railway are achieved.

Benefits of technology

It improved the structural integrity and texture fidelity of the 3D model, reduced the impact of environmental noise, reduced the amount of manual adjustment work, and achieved the reconstruction of a high-fidelity railway facility model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118973A_ABST
    Figure CN123118973A_ABST
Patent Text Reader

Abstract

This application relates to a method, apparatus, equipment, medium, and product for reconstructing a 3D model of a heavy-haul railway. The method includes: acquiring multi-view images of the target heavy-haul railway and determining corresponding initial scene metadata based on the multi-view images; training a 3D scene reconstruction model to be trained based on the multi-view images, the initial scene metadata, and preset prior constraints to obtain a trained initial 3D scene reconstruction model; acquiring pseudo-ground value images corresponding to the rendered images of the initial 3D scene reconstruction model, updating the initial 3D scene reconstruction model based on the pseudo-ground value images to obtain a denoised 3D scene reconstruction model; determining an explicit triangular mesh model based on the denoised 3D scene reconstruction model, and repairing the explicit triangular mesh model based on a pre-constructed Jacobian matrix deformation field to obtain a 3D model of the target railway reconstructed for the target heavy-haul railway. This method can remove environmental noise interference and accurately reconstruct the 3D model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, and in particular to a method, apparatus, equipment, medium and product for reconstructing a three-dimensional model of a heavy-haul railway. Background Technology

[0002] With the increasing demand for digital operation and maintenance of heavy-haul railway infrastructure, 3D reconstruction technology is gradually being applied to the digital modeling of railway tracks, roadbeds, overhead contact lines, and the surrounding environment. Traditional techniques typically obtain 3D models of railway scenes by acquiring multi-view images and combining them with 3D reconstruction methods.

[0003] Traditional 3D reconstruction and restoration methods for heavy-haul railways primarily rely on oblique photogrammetry and dense point cloud acquisition, with manual adjustment of mesh vertices for restoration. However, these methods are prone to affecting the texture quality and scene representation of 3D models when dealing with complex terrain and harsh environments along heavy-haul railways, and environmental noise interference is difficult to remove. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for reconstructing a three-dimensional model of a heavy-haul railway that can remove environmental noise interference and accurately reconstruct the three-dimensional model, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for reconstructing a three-dimensional model of a heavy-haul railway, including:

[0006] Acquire multi-view images of the target heavy-haul railway and determine the corresponding initial scene metadata based on the multi-view images; the initial scene metadata includes sparse point cloud, camera intrinsic parameters, and camera extrinsic parameters;

[0007] Based on the multi-view images, the initial scene metadata, and the preset prior constraints for the 3D scene reconstruction model to be trained, the 3D scene reconstruction model to be trained is trained to obtain the initial 3D scene reconstruction model after training.

[0008] Obtain the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model, and update the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain a denoised 3D scene reconstruction model.

[0009] Based on the denoised 3D scene reconstruction model, an explicit triangular mesh model is determined. Based on the pre-constructed Jacobian matrix deformation field, the explicit triangular mesh model is repaired to obtain a target railway 3D model for the reconstruction of the target heavy-haul railway.

[0010] In one embodiment, training the 3D scene reconstruction model to obtain a trained initial 3D scene reconstruction model based on the multi-view images, the initial scene metadata, and preset prior constraints for the 3D scene reconstruction model to be trained includes: obtaining a virtual observation viewpoint different from the acquisition viewpoint corresponding to the multi-view images based on the camera extrinsic parameters included in the initial scene metadata; obtaining the corresponding real viewpoint rendering image and virtual viewpoint rendering image under the acquisition viewpoint and the virtual observation viewpoint corresponding to the multi-view images, respectively; and training the 3D scene reconstruction model to obtain a trained initial 3D scene reconstruction model based on the real viewpoint rendering image, the virtual viewpoint rendering image, the multi-view images, and the preset prior constraints for the 3D scene reconstruction model to be trained.

[0011] In one embodiment, training the 3D scene reconstruction model to obtain a trained initial 3D scene reconstruction model based on the real-view rendering image, the virtual-view rendering image, the multi-view images, and preset prior constraints for the 3D scene reconstruction model to be trained includes: determining the pixel color difference between the real-view rendering image and the multi-view images based on the photometric reconstruction constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, and obtaining the corresponding photometric loss; determining the semantic differences between the image features of the virtual-view rendering image and the multi-view images based on the semantic topological invariance constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, and obtaining the corresponding semantic loss; and training the 3D scene reconstruction model to be trained based on the photometric loss and the semantic loss to obtain the trained initial 3D scene reconstruction model.

[0012] In one embodiment, obtaining the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model includes: obtaining a scene rendering image from the initial 3D scene reconstruction model under a preset viewpoint; and obtaining the pseudo-ground value image corresponding to the scene rendering image based on the scene rendering image.

[0013] In one embodiment, updating the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain a denoised 3D scene reconstruction model includes: replacing the image in the multi-view image corresponding to the viewpoint of the pseudo-ground value image based on the pseudo-ground value image to obtain an updated viewpoint image corresponding to the multi-view image; and updating the initial 3D scene reconstruction model based on the updated viewpoint image to obtain a denoised 3D scene reconstruction model.

[0014] In one embodiment, the step of determining an explicit triangular mesh model based on the denoised 3D scene reconstruction model, and repairing the explicit triangular mesh model based on a pre-constructed Jacobian matrix deformation field to obtain a target railway 3D model for the reconstruction of the target heavy-haul railway includes: determining the explicit triangular mesh model corresponding to the denoised 3D scene reconstruction model through a preset isosurface extraction algorithm; updating the pre-constructed Jacobian matrix deformation field based on the explicit triangular mesh model to obtain an updated Jacobian matrix deformation field; and repairing the explicit triangular mesh model based on the updated Jacobian matrix deformation field to obtain a target railway 3D model for the reconstruction of the target heavy-haul railway.

[0015] Secondly, this application also provides a device for reconstructing a three-dimensional model of a heavy-haul railway, comprising:

[0016] The data acquisition module is used to acquire multi-view images of the target heavy-haul railway and initial scene metadata corresponding to the multi-view images; the initial scene metadata is calculated based on the multi-view images using a preset calculation algorithm, including sparse point cloud, camera intrinsic parameters and camera extrinsic parameters.

[0017] The initial modeling module is used to train the 3D scene reconstruction model to be trained based on the multi-view images, the initial scene metadata, and the preset prior constraints for the 3D scene reconstruction model to be trained, so as to obtain the trained initial 3D scene reconstruction model.

[0018] The reverse denoising module is used to obtain the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model, and update the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain the denoised 3D scene reconstruction model.

[0019] The repair output module is used to repair the denoised 3D scene reconstruction model based on the explicit triangular mesh model determined by the denoised 3D scene reconstruction model and the pre-constructed Jacobian matrix deformation field, to obtain a target railway 3D model reconstructed for the target heavy-haul railway.

[0020] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method steps of any one of the first aspects.

[0021] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method steps of any one of the first aspects.

[0022] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the method steps of any one of the first aspects.

[0023] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for reconstructing a 3D model of a heavy-haul railway acquire multi-view images of the target heavy-haul railway and initial scene metadata including sparse point clouds, camera intrinsic parameters, and camera extrinsic parameters. Based on these multi-view images, initial scene metadata, and preset prior constraints, the 3D scene reconstruction model is trained. This allows the 3D scene reconstruction model to learn the scene structure of the target heavy-haul railway under the combined influence of image information, spatial geometric information, and prior constraints, thereby improving the structural integrity of the 3D scene reconstruction under sparse observation conditions. By acquiring pseudo-ground value images corresponding to the rendered images of the initial 3D scene reconstruction model and updating the initial 3D scene reconstruction model based on these pseudo-ground value images, the denoised image information is used to influence the 3D scene reconstruction process, thus reducing the impact of environmental noise on the 3D model. Finally, by determining an explicit triangular mesh model based on the denoised 3D scene reconstruction model and repairing the explicit triangular mesh model using a Jacobian matrix deformation field, a 3D model of the target railway reconstructed for the target heavy-haul railway is obtained. This achieves the repair of incomplete or erroneous geometric structures in the 3D model of the heavy-haul railway and preserves the high-fidelity texture of the denoised model in the repaired model. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is an application environment diagram of a method for reconstructing a three-dimensional model of a heavy-haul railway in one embodiment;

[0026] Figure 2 This is a flowchart illustrating a method for reconstructing a three-dimensional model of a heavy-haul railway in one embodiment.

[0027] Figure 3 This is a schematic diagram of the architecture of a dedicated image semantic feature encoding model for heavy-haul railway scenarios in one embodiment;

[0028] Figure 4This is a schematic diagram illustrating the construction process of the initial 3D scene reconstruction model in one embodiment;

[0029] Figure 5 This is a schematic diagram of the construction process for building a generative diffusion model in one embodiment;

[0030] Figure 6 This is a flowchart illustrating an explicit geometric repair mechanism based on a Jacobian matrix field in one embodiment.

[0031] Figure 7 This is a flowchart illustrating a method for reconstructing a three-dimensional model of a heavy-haul railway in another embodiment;

[0032] Figure 8 This is a structural block diagram of a device for reconstructing a three-dimensional model of a heavy-haul railway in one embodiment;

[0033] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0036] Heavy-haul railways, as an important transportation corridor in my country, traverse complex terrains, crossing various geographical environments including mountains, bridges, and tunnels. Constructing high-fidelity 3D railway scene models is crucial not only for the digital operation and maintenance and defect monitoring of railway infrastructure, but also for achieving refined monitoring of railway energy consumption, full life-cycle management of carbon emissions, and green and low-carbon operational decisions. This is of great significance for promoting the green transformation of the railway transportation industry.

[0037] Currently, railway 3D reconstruction and restoration technologies primarily rely on traditional oblique photogrammetry and dense point cloud acquisition for reconstruction, followed by manual adjustment of mesh vertices for restoration. Oblique photogrammetry typically involves mounting multiple sensors on a flight platform to simultaneously acquire images from vertical and multiple oblique angles, using aerial triangulation and texture mapping techniques to reconstruct a 3D model of the ground surface. Dense point cloud acquisition mainly relies on lidar equipment, which actively emits laser pulses and receives echoes to directly obtain high-precision 3D spatial coordinate point sets of the target object's surface. In recent years, with the development and breakthroughs in deep learning technology, 3D reconstruction technologies, represented by neural radiation fields and large-scale model reconstruction, have gradually been introduced. Neural radiation fields, as a 3D scene construction method based on implicit neural representation, utilizes a multilayer perceptron to map spatial coordinates and viewing angles into volume density and color, synthesizing high-quality new perspective images through volume rendering techniques along ray integration. The reconstruction large model is a 3D reconstruction large model based on the Transformer architecture. It usually adopts a feedforward mechanism, learns general 3D geometric priors by pre-training on a large-scale 3D dataset, and uses a cross-attention mechanism to project and map 2D image features into 3D features, thus having the ability to quickly infer and generate 3D models from single or sparse images within seconds.

[0038] However, the aforementioned 3D scene reconstruction and restoration technologies still have shortcomings and limitations when facing complex field environments such as heavy-haul railways. First, there is the problem of sparse view reconstruction failure due to limited data acquisition. The environment along heavy-haul railways is harsh, and drones or inspection vehicles, limited by flight safety distances and battery life, often struggle to acquire densely packed, highly overlapping, and fully covered view images. Traditional NeRF (Neural Radiance Fields) methods heavily rely on dense input views; under sparse views, due to the lack of geometric constraints, they are prone to overfitting, leading to severe blurring, artifacts, or geometric collapse in the rendering of new views. While existing regularization methods can alleviate some artifacts, they often sacrifice high-frequency details, failing to meet the requirements for refined reconstruction of key railway components. Second, there is the problem of poor scene semantic consistency caused by unstructured environmental noise. Railway field scenes are filled with environmental interference such as weed obstruction, uneven lighting, and weather changes. Traditional reconstruction methods reconstruct this interference information as part of the real geometry, lacking the ability to semantically understand and filter scene content. While some methods attempt to incorporate diffusion models for image editing, directly denoising or stylizing single-frame images easily disrupts geometric consistency across multiple viewpoints. This results in flickering or structural misalignment in the synthesized 3D model from different angles, making it impossible to construct a clean and standardized railway facility model. Furthermore, there is a lack of precise geometric repair and deformation capabilities based on semantic instructions. Traditional geometric repair relies on manual adjustment of mesh vertices, which is not only extremely labor-intensive but also easily disrupts the smoothness of the model surface, leading to topological inconsistencies.

[0039] The method for reconstructing a three-dimensional model of a heavy-haul railway provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 is used to acquire multi-view images of the target heavy-haul railway and determine the corresponding initial scene metadata based on the multi-view images; the initial scene metadata includes sparse point clouds, camera intrinsic parameters, and camera extrinsic parameters; based on the multi-view images, initial scene metadata, and preset prior constraints for the 3D scene reconstruction model to be trained, the 3D scene reconstruction model to be trained is trained to obtain a trained initial 3D scene reconstruction model; the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model is acquired, and based on the pseudo-ground value image, the initial 3D scene reconstruction model is updated to obtain a denoised 3D scene reconstruction model; based on the explicit triangular mesh model determined by the denoised 3D scene reconstruction model and the pre-constructed Jacobian matrix deformation field, the denoised 3D scene reconstruction model is repaired to obtain the target railway 3D model reconstructed for the target heavy-haul railway. Among them, terminal 102 can be, but is not limited to, various personal computers, laptops, tablets, and server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0040] In one exemplary embodiment, such as Figure 2 As shown, a method for reconstructing a three-dimensional model of a heavy-haul railway is provided, and this method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 208. Wherein:

[0041] Step 202: Obtain multi-view images of the target heavy-haul railway and determine the corresponding initial scene metadata based on the multi-view images; the initial scene metadata includes sparse point cloud, camera intrinsic parameters and camera extrinsic parameters.

[0042] The target heavy-haul railway refers to the heavy-haul railway line to be reconstructed in 3D, along with its railway facilities and surrounding environment, which may include the railway subgrade, rails, sleepers, fasteners, and overhead contact line suspension devices. Multi-view images refer to images of the target heavy-haul railway scene acquired from different acquisition locations and observation perspectives, used to describe the spatial structure and surface information of the target heavy-haul railway from multiple perspectives. In this embodiment, drones and track inspection vehicles can be used to acquire images along the target heavy-haul railway line, either separately or collaboratively, to obtain multi-view images including the railway subgrade, track, overhead contact line, and surrounding environment. Initial scene metadata is the scene geometric information corresponding to the multi-view images, used to describe the spatial positional relationships corresponding to the multi-view images and the initial 3D structure of the target heavy-haul railway scene. In this embodiment, the initial scene metadata includes sparse point clouds, camera intrinsic parameters, and camera extrinsic parameters; the sparse point cloud is used to characterize the initial spatial structure of the target heavy-haul railway scene determined by the multi-view images; the camera intrinsic parameters are used to characterize the imaging parameters of the camera that acquired the multi-view images; and the camera extrinsic parameters are used to characterize the position and orientation of the camera in the target heavy-haul railway scene when acquiring the multi-view images.

[0043] For example, firstly, the server acquires multi-view images along the planned heavy-haul railway line from low-altitude drones and track inspection vehicles equipped with high-resolution cameras. These multi-view images cover the target heavy-haul railway's roadbed, track, overhead contact system, and surrounding environment. Subsequently, a pre-defined railway feature geometric constraint calculation algorithm is used to calculate the initial metadata of the multi-view image sequence. This initial metadata includes the camera intrinsic parameter matrix, camera extrinsic parameters, and sparse point clouds.

[0044] Step 204: Based on multi-view images, initial scene metadata, and preset prior constraints for the 3D scene reconstruction model to be trained, train the 3D scene reconstruction model to be trained to obtain the trained initial 3D scene reconstruction model.

[0045] The 3D scene reconstruction model to be trained is used to establish a 3D scene representation of the target heavy-haul railway based on multi-view images and initial scene metadata. Preset prior constraints are constraints pre-set to constrain the training process of the 3D scene reconstruction model to be trained, used to constrain the scene structure of the target heavy-haul railway when the number of multi-view images is limited. In this embodiment, the preset prior constraints mainly include photometric reconstruction constraints based on real-view rendering results and semantic topological invariance constraints based on virtual observation views. The initial 3D scene reconstruction model is the model obtained after training the 3D scene reconstruction model to be trained, used to characterize the 3D scene structure and surface information of the target heavy-haul railway.

[0046] For example, the server first constructs an implicit 3D scene reconstruction network framework adapted to the sparse observation perspective of heavy-haul railways based on preset prior constraints. Then, initial scene metadata, such as multi-view images, camera poses, intrinsic parameters, and extrinsic parameters, is input into the constructed implicit 3D scene reconstruction network framework. The implicit 3D scene reconstruction network utilizes the preset prior constraints to perform iterative optimization and parameter solving. Under the condition that preset training completion conditions are met, it outputs a structurally complete, trained initial implicit 3D model of the railway scene.

[0047] Step 206: Obtain the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model, and update the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain the denoised 3D scene reconstruction model.

[0048] The rendered image refers to the image obtained by rendering the target heavy-haul railway scene using the initial 3D scene reconstruction model according to a preset observation viewpoint. It represents the scene appearance of the initial 3D scene reconstruction model from the corresponding viewpoint. The pseudo-ground truth image refers to the image obtained by denoising and editing the rendered image, used to replace the original observation image in updating the 3D scene reconstruction model. The pseudo-ground truth image is not a directly acquired real image, but rather an image obtained by processing the rendered image. The denoised 3D scene reconstruction model refers to the 3D scene reconstruction model obtained by updating the initial 3D scene reconstruction model using the pseudo-ground truth image.

[0049] For example, the server obtains a pseudo-ground value image corresponding to the scene rendering image under a preset viewpoint from the initial 3D scene reconstruction model, and then inputs the pseudo-ground value image into the training set of the initial 3D scene reconstruction model to update the initial 3D scene reconstruction model and obtain a denoised 3D scene reconstruction model.

[0050] Step 208: Determine the explicit triangular mesh model based on the denoised 3D scene reconstruction model, and repair the explicit triangular mesh model based on the pre-constructed Jacobian matrix deformation field to obtain the target railway 3D model for reconstruction of the target heavy-haul railway.

[0051] The explicit triangular mesh model refers to the explicit geometric representation of a 3D scene composed of vertices and triangular faces, used to directly describe the surface geometry of the target heavy-haul railway scene. In this embodiment, the explicit triangular mesh model is obtained by converting a denoised 3D scene reconstruction model. The Jacobian matrix deformation field is a deformation field used to describe the local geometric deformation of the explicit triangular mesh model. The target railway 3D model refers to the target heavy-haul railway 3D model obtained after environmental noise removal and geometric structure repair.

[0052] For example, the server determines an explicit triangular mesh model based on the denoised 3D scene reconstruction model, and then repairs and updates the explicit triangular mesh model according to the local deformation represented by the pre-constructed Jacobian matrix deformation site, to obtain the target railway 3D model for the reconstruction of the target heavy-haul railway.

[0053] In the aforementioned method for reconstructing a 3D model of a heavy-haul railway, the server first determines initial scene metadata, including sparse point clouds, camera intrinsic parameters, and camera extrinsic parameters, based on multi-view images of the target heavy-haul railway. This provides image information and initial spatial geometric information for 3D scene reconstruction. Subsequently, the server trains the 3D scene reconstruction model using multi-view images, initial scene metadata, and preset prior constraints. This allows the 3D scene reconstruction model to supplement scene geometric information under sparse observation conditions using real observation information and prior constraints, thereby improving the structural integrity of the target heavy-haul railway's 3D scene and reducing reconstruction blur, artifacts, and missing geometric structures caused by limited acquisition perspectives. Next, the server obtains pseudo-ground value images corresponding to the rendered images of the initial 3D scene reconstruction model and uses these pseudo-ground value images to reconstruct the initial 3D scene. The scene reconstruction model is updated so that the image information processed by environmental noise has a reverse effect on the 3D scene reconstruction process. This reduces the impact of unstructured environmental noise on the 3D model and reduces the structural inconsistencies between different viewpoints after denoising a single frame image. As a result, a denoised 3D scene reconstruction model with high surface texture fidelity is obtained. Finally, the server determines an explicit triangular mesh model based on the denoised 3D scene reconstruction model and repairs the explicit triangular mesh model using the Jacobian matrix deformation field. This yields a target railway 3D model for the reconstruction of the target heavy-haul railway. This achieves the repair of incomplete or erroneous geometric structures in the railway 3D model, reduces the repair operations required by traditional manual adjustment of mesh vertices, and preserves the high-fidelity surface texture of the denoised 3D scene reconstruction model in the repaired model.

[0054] In an exemplary embodiment, the training of the 3D scene reconstruction model to be trained, based on multi-view images, initial scene metadata, and preset prior constraints for the 3D scene reconstruction model to be trained, to obtain a trained initial 3D scene reconstruction model, includes: obtaining a virtual observation viewpoint different from the acquisition viewpoint corresponding to the multi-view images based on the camera extrinsic parameters included in the initial scene metadata; obtaining the corresponding real viewpoint rendering image and virtual viewpoint rendering image under the acquisition viewpoint and virtual observation viewpoint corresponding to the multi-view images, respectively; and training the 3D scene reconstruction model to be trained based on the real viewpoint rendering image, virtual viewpoint rendering image, multi-view images, and preset prior constraints for the 3D scene reconstruction model to be trained, to obtain a trained initial 3D scene reconstruction model.

[0055] In this context, the acquisition viewpoint refers to the camera's observation viewpoint corresponding to the acquisition of multi-view images, which can be determined based on the camera extrinsic parameters in the initial scene metadata. The virtual observation viewpoint refers to a scene observation viewpoint different from the acquisition viewpoint corresponding to the multi-view images, used for virtual observation of the target heavy-haul railway scene outside the original image acquisition viewpoint. The virtual observation viewpoint is not the actual acquired viewpoint, but rather a viewpoint determined after perturbing the viewpoint based on existing camera extrinsic parameters. The real-view rendering image refers to the image obtained by rendering the scene under the acquisition viewpoint corresponding to the multi-view images using the 3D scene reconstruction model to be trained. The virtual-view rendering image refers to the image obtained by rendering the scene under the virtual observation viewpoint using the 3D scene reconstruction model to be trained.

[0056] Among them, the preset prior constraints are determined based on a special image semantic feature encoding model for heavy-haul railway scenarios, such as... Figure 3 As shown, the image semantic feature encoding model adopts a semantic feature-aligned dual-stream parallel network architecture, including an image encoder branch and a text encoder branch. The image encoder branch includes an image block embedding and position encoding layer, a 12-layer stacked encoder layer, and a visual mapping head. The image block embedding and position encoding layer performs image block segmentation, linear mapping, and position encoding on the input railway scene image I to obtain an image block sequence vector. Each encoder layer includes layer normalization, multi-head self-attention, layer normalization, and a multilayer perceptron, and residual connections can be set. The 12-layer encoder layer outputs global visual features, which are mapped to image semantic feature vectors by the visual mapping head. The text encoder branch includes a word embedding and position encoding layer, a 12-layer stacked encoder layer, and a text mapping head. The token embedding and positional encoding layer performs word segmentation, word embedding, and positional encoding on the input text to obtain a token sequence vector. Each encoder layer sequentially includes layer normalization (Norm), multi-head self-attention, layer normalization, and a multilayer perceptron, and residual connections can be set. The 12 encoder layers output global text features, which are mapped to text semantic feature vectors by a text mapping head. Both the visual mapping head and the text mapping head are non-linear projection heads used to map the image semantic feature vectors and text semantic feature vectors to a joint semantic feature space of the same dimension, and can be normalized. Subsequently, the loss calculation module receives the normalized image semantic feature vectors and text semantic feature vectors, calculates the image-to-text contrastive loss and the text-to-image contrastive loss, and obtains the total contrastive loss to optimize the semantic feature alignment network parameters.

[0057] The image semantic feature encoding model can be trained through the following steps: First, a semantic feature alignment dual-stream parallel network architecture is constructed, which includes an image encoder. With text encoder The image encoder comprises a linear projection layer, a positional encoding layer (image patch embedding and positional encoding), and a multi-layer self-attention module (encoder layer). The image encoder segments the input multi-view image (railway scene image) I into fixed-size patches (image blocks), maps them to sequence vectors through the linear projection layer (linear mapping), and adds positional encoding (adding positional encoding). Finally, it inputs the multi-layer self-attention module (encoder layer) to generate global visual features (image semantic feature vector). The text semantic encoder comprises a word embedding layer, a positional encoding layer (word embedding and positional encoding), and a stacked multi-head self-attention module (encoder layer). The text encoder first segments the input text into words and converts it into a discrete index sequence (word segmentation). Then, it maps the word embedding layer to a high-dimensional dense vector and superimposes positional encoding (word embedding). Finally, it inputs the multi-layer self-attention module (encoder layer) to capture long-distance dependencies in the text sequence, generating global text semantic features (text semantic feature vector). Finally, non-linear projection heads are designed at the ends of the two branches to transform the image semantic feature vector (image features). Text semantic feature vectors (text features) Mapped to the same dimension of joint semantic feature space Subsequently, a selection of typical railway scene images were chosen from multi-view images, including samples with varying lighting, weather, and viewing angles. This ensured the model could learn the essential semantic features of key facilities such as "rails" and "sleepers" under different environments. Corresponding text descriptions were then added to each typical railway scene image, constructing a system containing... A dataset of railway-specific images and texts containing 10 sample pairs. Next, the railway-specific image and text dataset is input into the semantic feature alignment network for contrastive learning training, using a loss function. Optimize the semantic feature alignment network parameters to make the dot product of feature vectors Approaching 1 on positive sample pairs, maximizing paired samples. Cosine similarity in the feature space, while minimizing the similarity between unpaired samples, enables the semantic feature alignment network to understand the deep association between the visual morphology and linguistic semantics of railway facilities. Loss function Specifically as follows:

[0058]

[0059] in For image-to-text contrast loss, This represents the contrast loss from text to image.

[0060] Image-to-text contrast loss Specifically as follows:

[0061]

[0062] The text-to-image contrast loss Specifically as follows:

[0063]

[0064] in, The training batch size is the number of image-text pairs input in one iteration. and These are the indices of the current anchor sample and the comparison sample, respectively. and The first Zhang Railway Scene Image and the First The image feature vector and text feature vector of each text description after being mapped and normalized by their respective encoders. The cosine similarity function measures the similarity between image and text features in the semantic space. This is a learnable temperature coefficient scalar used to control the smoothness of the Softmax distribution and adjust the model's focus on hard-negative samples. To convert similarity into an exponential function of non-negative probability values, the summation in the denominator... This is the sum of similarities between the current sample and all samples in the batch, used to calculate the normalized probability distribution. The image encoder is frozen after training convergence. The parameters are used to obtain an adapted railway image semantic feature encoding model, which is then used as a fixed semantic supervisor for neural radiation field optimization in subsequent steps.

[0065] For example, the server first constructs an implicit field mapping network adapted to the sparse observation perspective of heavy-haul railways and used to characterize the complex spatial geometry of heavy-haul railways, based on preset prior constraints for the 3D scene reconstruction model to be trained. ,like Figure 4 As shown, the server inputs the sparse multi-view image I and camera extrinsic parameters (R,t) from the initial scene metadata into the view sampling module. The view sampling module includes real-view ray sampling and virtual view generation: real-view ray sampling performs ray sampling on the real acquisition viewpoint based on the camera extrinsic parameters; virtual view generation uses the camera pose corresponding to the camera extrinsic parameters (R,t) as a reference, applies random rotation and / or translational perturbations to generate a virtual observation viewpoint different from the original acquisition path, and performs ray sampling on the virtual observation viewpoint. The implicit neural radiation field network performs high-frequency position encoding on the spatial coordinates and / or viewpoint direction of the sampling points, inputting them into the density branch MLP (Multi-Layer Perceptron) and color branch MLP respectively to obtain the volume density. And color c. The volume renderer performs volume rendering integration along the light rays to synthesize the real-view rendering map and the virtual-view rendering map. Then, the photometric rendering fidelity constraint loss is calculated by combining the real-view rendering map with the ground truth values ​​of the real images in the multi-view images; the virtual-view rendering map and the real-view images are respectively input into the frozen semantic feature encoder to extract image semantic features and calculate the cross-view semantic topology invariance loss. The photometric rendering fidelity constraint loss and the cross-view semantic topology invariance loss are weighted and summed to obtain the total loss function. Based on the total loss function, the parameters of the implicit neural radiation field network are optimized by backpropagation until convergence, and a high-fidelity railway neural radiation field is output, which is the initial 3D scene reconstruction model after training.

[0066] Specifically, the implicit field mapping network corresponds to the implicit neural radiation field network in the figure. This implicit field mapping network consists of a high-frequency position encoding module (position encoding), a density prediction multilayer perceptron branch, and a color prediction multilayer perceptron branch (color branch MLP). Among them, the density prediction multilayer perceptron branch can be called the density branch MLP. Volume density can be obtained through the density branch MLP. The color c can be obtained through the color branch MLP. The server uses the camera pose corresponding to the camera extrinsic parameters (R,t) as a reference, and applies preset rotation and / or translation perturbations to the camera pose to obtain a virtual observation viewpoint different from the original acquisition path. After obtaining the virtual observation viewpoint, the 3D scene reconstruction model to be trained is used to render the scene under the acquisition viewpoints corresponding to the multi-view images and the virtual observation viewpoints, respectively, to obtain the corresponding real-view rendering images and virtual-view rendering images. Subsequently, based on the real-view rendering images, virtual-view rendering images, multi-view images, and preset prior constraints, the 3D scene reconstruction model to be trained is trained. The image semantic feature encoding model is used as the prior constraints provided by the semantic supervisor for iterative optimization and parameter solving. The model parameters of the 3D scene reconstruction model to be trained are updated iteratively until the preset convergence condition is met, resulting in the initial 3D scene reconstruction model after training.

[0067] The implicit neural radiation field in this embodiment is only an example of a network and can also be transferred to hybrid representation schemes including 3DGaussian Splatting (3DGS) or SDF (Signed Distance Function).

[0068] In this embodiment, by generating a virtual observation perspective that differs from the actual acquisition perspective based on camera extrinsic parameters, and simultaneously acquiring both the real and virtual viewpoint renderings, the observation range of the 3D scene reconstruction model can be expanded when the actual acquisition perspective is limited. This allows the model to obtain scene information beyond the original acquisition perspective using the virtual observation perspective, thereby reducing the loss of scene geometric information caused by the limited acquisition of actual images of heavy-haul railways and the sparse observation perspective, and improving the structural integrity of the initial 3D scene reconstruction model.

[0069] In an exemplary embodiment, the 3D scene reconstruction model to be trained is trained based on a real-view rendering image, a virtual-view rendering image, multi-view images, and preset prior constraints for the 3D scene reconstruction model to be trained, to obtain a trained initial 3D scene reconstruction model; this includes: determining the pixel color difference between the real-view rendering image and the multi-view images based on the photometric reconstruction constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, and obtaining the corresponding photometric loss; determining the semantic differences between the image features of the virtual-view rendering image and the multi-view images based on the semantic topological invariance constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, and obtaining the corresponding semantic loss; and training the 3D scene reconstruction model to be trained based on the photometric loss and the semantic loss, to obtain the trained initial 3D scene reconstruction model.

[0070] The photometric reconstruction constraint is used to ensure the consistency of image content between the real-view rendered image and the corresponding multi-view images. It compares the pixel color differences at corresponding image locations to ensure the 3D scene reconstruction model restores pixel details from the actual observed images. Photometric loss represents the amount of loss due to pixel color differences between the real-view rendered image and the corresponding multi-view images; the smaller the pixel color differences, the smaller the photometric loss. The semantic topological invariance constraint is used to ensure the consistency of semantic features of the target heavy-load railway scene under different observation perspectives. Semantic loss represents the amount of loss due to the difference between the semantic features corresponding to the virtual view rendered image and the semantic features corresponding to the multi-view images.

[0071] For example, the server performs pixel-level comparison between the real-view rendering image and its corresponding multi-view image. Based on the photometric reconstruction constraint in the preset prior constraints for the 3D scene reconstruction model to be trained, it calculates the color difference between corresponding pixel positions and determines the photometric loss based on the color difference at each pixel position. Subsequently, based on the semantic topological invariance constraint in the preset prior constraints for the 3D scene reconstruction model to be trained (i.e., the prior constraints provided by the image semantic feature encoding model as a semantic supervisor), it encodes the virtual-view rendering image and the multi-view image respectively, obtaining the semantic features corresponding to the virtual-view rendering image and the semantic features corresponding to the multi-view image, and determines the semantic loss based on the similarity between the two types of semantic features.

[0072] Specifically, such as Figure 4 As shown, the server first constructs the total loss function. To drive the implicit field network for parameter updates, the total loss function is... Specifically as follows:

[0073]

[0074] in This refers to the photometric loss (photometric rendering fidelity constraint loss). This is semantic loss (cross-perspective semantic topology invariance loss). These are the weighting coefficients.

[0075] During training, the acquired multi-view images are rendered using a volume renderer to obtain a real-view rendering image. Based on the real-view rendering image and the multi-view images, the photometric loss (photometric rendering fidelity constraint loss) is calculated to constrain the network to restore the pixel details of the image. At the same time, by using the image encoder in the image semantic feature encoding model as a semantic supervisor (semantic encoder (frozen)), the semantic loss between the virtual view rendering image and the real scene features (multi-view image features) (cross-view semantic topological invariance loss) is calculated. The semantic loss is used as a high-dimensional semantic prior constraint to optimize the network parameters until the model converges. Finally, the initial 3D scene reconstruction model (neural radiation field) is trained.

[0076] The specific optical loss and semantic loss are as follows:

[0077]

[0078]

[0079] in The pixel color predicted by the implicit field mapping network (implicit neural radiation field network). This represents the true color value of the corresponding pixel in the real image. A collection of virtual perspectives The cosine similarity function is used. Image encoder, Rendering images from a virtual perspective. These are multi-view images (real images).

[0080] In this embodiment, by using photometric reconstruction constraints to determine photometric loss, the pixel color difference between the real-view rendering and the multi-view images can be used to constrain the 3D scene reconstruction model, making the model more accurately reproduce the image details in the actual acquired images. At the same time, by using semantic topological invariance constraints to determine semantic loss, the semantic feature relationship between the virtual-view rendering and the multi-view images can be used to semantically constrain the scene structure of the model under the actual acquired view.

[0081] In an exemplary embodiment, obtaining a pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model includes: obtaining a scene rendering image from the initial 3D scene reconstruction model under a preset viewpoint; and obtaining a pseudo-ground value image corresponding to the scene rendering image based on the scene rendering image.

[0082] The preset viewpoint refers to the pre-determined observation viewpoint used to obtain the scene rendering image from the initial 3D scene reconstruction model. It can be any viewpoint among the acquisition viewpoints corresponding to the multi-view image, or it can be selected sequentially from multiple acquisition viewpoints according to actual processing needs. The scene rendering image refers to the image obtained by rendering the target heavy-haul railway scene using the initial 3D scene reconstruction model according to the preset viewpoint. The pseudo-ground truth image refers to the image obtained after image editing based on the scene rendering image, used as a temporary ground truth to participate in the updating of the 3D scene reconstruction model. Unlike the actual acquired real image, the pseudo-ground truth image is an image generated or edited through the model.

[0083] Among them, such as Figure 5As shown, the architecture of the generative diffusion model is as follows: the inputs are image conditions and text instructions; the text instructions are input to the text encoder, and the image conditions are input to the image encoder, which encodes them into the latent space to obtain noise latent variables; the semantic instruction injection module injects the high-dimensional semantic embedding vector output by the text encoder into each feature layer of the hierarchical feature denoising backbone network; the structural geometry coupling module extracts spatial structural features such as rail alignment and roadbed contour in the railway image to be edited, and physically concatenates them with the noise latent variables in the channel dimension as input to the hierarchical feature denoising backbone network; the hierarchical feature denoising backbone network includes a downsampling module, an intermediate bottleneck layer, and an upsampling module, wherein the downsampling module includes 12 downsampling layers, which are used to progressively reduce the spatial resolution of the feature map and extract multi-scale semantic features, and the upsampling module includes 12 upsampling layers, which are used to progressively restore the spatial resolution of the feature map and perform skip connections and fusion with the features of the corresponding layer in the downsampling stage; the feature map is progressively reduced through the downsampling module. The model extracts multi-scale semantic features by resolving spatial resolution. Noise prediction and conditional feature fusion are performed in the low-resolution feature space through an intermediate bottleneck layer. An upsampling module progressively restores the spatial resolution of the feature map and fuses it with features from the corresponding layers in the downsampling stage via skip connections. Both the downsampling and upsampling modules include residual networks, self-attention modules, and cross-attention modules. The residual network is used to construct residual connections to stabilize training and preserve multi-scale features. The self-attention module is used to model long-distance dependencies within the feature map. The cross-attention module is used to fuse text semantic embedding vectors and structural geometric conditions into the backbone network features. Simultaneously, it combines temporal embedding for noise prediction and reverse denoising sampling, ultimately decoding and outputting the repaired image. During training, this generative diffusion model uses a triplet instruction containing disturbed state images, text instructions, and clean ground truth images to drive the image editing dataset, and is trained under supervision using the railway unstructured noise latent space prediction residual loss function. Specifically, in this embodiment, editing of pseudo-true images can be achieved through a generative diffusion model that fuses textual instructions and image conditions to address the unstructured environmental noise of heavy-haul railways. The construction of this generative diffusion model first requires building a multimodal conditional generative diffusion network architecture adapted to the complex texture features of heavy-haul railways. The diffusion network architecture mainly consists of a hierarchical feature denoising backbone network, a semantic instruction injection module, and a structural geometry coupling module. The semantic instruction injection module and the structural geometry coupling module serve as interfaces to the backbone network. The semantic instruction injection module uses a text encoder to map natural language editing instructions into high-dimensional semantic embedding vectors and embeds the semantic features into various feature layers of the backbone network. The structural geometry coupling module is responsible for extracting the spatial structural features of the railway image to be edited. It physically concatenates the extracted spatial structural features with latent noise variables in the channel dimension as input to the backbone network, thereby forcing the network to strictly preserve the rigid structural information of the original scene, such as the rail alignment and roadbed contour, during the generation process. A portion of high-quality, unobstructed railway images from the acquired multi-view images are selected as ground truth images. By directionally superimposing vegetation occlusion, rain / fog textures, or Gaussian noise onto the ground truth image, a corresponding disturbance state image is generated. A large-scale instruction-driven image editing dataset was constructed based on two types of data: ground truth images and images in perturbation state, along with text descriptions. This dataset contains a large number of triplet samples. ,in Images containing "disturbed conditions" such as weeds, stagnant water, or blurred textures. This is the corresponding ground truth image after removing interference. The text instructions that describe the image editing operation.

[0084] The generation of railway interference status images is based on the following calculation formula:

[0085]

[0086] in The original, clean ground truth image. For the noise source image used to simulate interference, This is a mask matrix used to define the spatial distribution region of interference in the image. This indicates element-wise multiplication.

[0087] Design and construct a residual loss function for the latent spatial prediction of unstructured noise in railways. Using image editing datasets The network model is trained iteratively using the image editing dataset. Input to the network generative diffusion network Supervised training is performed to optimize the parameters of the generative diffusion network using the loss function until the loss function is optimized. The model converges, resulting in a well-trained generative diffusion model.

[0088] loss function Specifically as follows:

[0089]

[0090] in To make the truth image The latent variables are encoded into the latent space and then superimposed with noise corresponding to the time step. For time steps, It is Gaussian noise. For network prediction noise, For image encoders, Images containing "disturbed conditions" such as weeds, stagnant water, or blurred textures. The text instruction that describes the editing operation.

[0091] For example, such as Figure 5 As shown, the server connects the initial 3D scene reconstruction model with the generative diffusion model, and randomly selects a camera pose from the multi-view images. This involves obtaining the camera pose from a preset viewpoint, reconstructing the current 3D scene model using that pose, and then performing volume rendering to generate the current scene rendering image. Simultaneously, if the current scene rendering image has occlusions, a natural language denoising instruction is generated based on the specific occlusion content. For example, if weeds are present, a natural language denoising instruction of "clear the weeds from the roadbed surface" is generated, and this instruction is input into a text encoder to generate a text embedding vector. A diffusion editing network is then used to process the input scene rendering image. Perform semantically guided image editing. To prevent damage to the original rigid geometry of railway facilities, such as rails and overhead contact line supports, during denoising, a hybrid noise injection strategy is adopted. After encoding into the latent space, random Gaussian noise is mixed with the original image structural features, and then used in the text instructions. Guided by [the desired method], reverse denoising sampling is performed to generate a pseudo-ground value image that meets denoising requirements while preserving the original geometric contours. .

[0092] The mathematical expression for the editing process is as follows:

[0093]

[0094] in For latent variables that combine original image features with noise, It is a generative diffusion network.

[0095] In this embodiment, by obtaining scene rendering images from a preset viewpoint from the initial 3D scene reconstruction model and generating pseudo-ground images based on these rendering images, image data processed by environmental noise can be obtained without re-acquiring images of the target heavy-haul railway. This data can then be used as reference information for updating the subsequent 3D scene reconstruction model. Simultaneously, by employing a generative diffusion model to edit the scene rendering images, unstructured environmental disturbances such as weeds, rain, fog, water accumulation, and blurred textures in the heavy-haul railway scene can be addressed. Furthermore, by preserving the original image structural features, the impact of image editing on railway facility structural information such as rails, catenary supports, and roadbed contours is reduced, thus providing image information with high structural fidelity for subsequent 3D scene model denoising.

[0096] In an exemplary embodiment, updating the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain a denoised 3D scene reconstruction model includes: replacing the image in the multi-view image corresponding to the viewpoint of the pseudo-ground value image based on the pseudo-ground value image to obtain the updated viewpoint image corresponding to the multi-view image; and updating the initial 3D scene reconstruction model based on the updated viewpoint image to obtain the denoised 3D scene reconstruction model.

[0097] Among them, the denoised 3D scene reconstruction model refers to the 3D scene reconstruction model obtained after multiple rounds of pseudo-true value image replacement and model parameter update, which is used to characterize the target heavy-haul railway scene after removing unstructured environmental noise.

[0098] For example, the server generates a pseudo-truth image. As a temporary ground truth, it replaces the corresponding pose in the multi-view image. The observation data was then used. Then, an implicit field mapping network was employed. An implicit field texture evolution consistency loss function is constructed, and the parameters of the implicit field mapping network are updated through the loss function until the loss function converges. This step is repeated on all original images to update the initial 3D scene reconstruction model until the rendering results of all views in the scene have achieved environmental noise removal while maintaining multi-view consistency, resulting in a denoised 3D scene reconstruction model.

[0099] loss function for:

[0100]

[0101] in, Indicates camera pose The collection of light rays below, The pixel colors predicted for the initial 3D scene reconstruction model.

[0102] In this embodiment, by replacing the image corresponding to the viewpoint in the multi-view image with a pseudo-ground value image, an updated viewpoint image is obtained. This allows the image information processed by environmental noise to be reintroduced into the 3D scene reconstruction process, so that the 3D scene reconstruction model no longer continuously uses the original observation data containing environmental noise for scene representation. By updating the initial 3D scene reconstruction model based on the updated viewpoint image and repeating the above process for multiple viewpoints, the processed viewpoint images can gradually act in reverse on the 3D scene reconstruction model, allowing the model to gradually remove unstructured environmental noise such as weeds, rain, and fog. At the same time, the multi-view representation of the 3D scene model itself is used to maintain scene consistency between different viewpoints.

[0103] In an exemplary embodiment, an explicit triangular mesh model is determined based on a denoised 3D scene reconstruction model. Based on a pre-constructed Jacobian matrix deformation field, the explicit triangular mesh model is repaired to obtain a target railway 3D model for the reconstruction of the target heavy-haul railway. This includes: determining the explicit triangular mesh model corresponding to the denoised 3D scene reconstruction model using a preset isosurface extraction algorithm; updating the pre-constructed Jacobian matrix deformation field based on the explicit triangular mesh model to obtain an updated Jacobian matrix deformation field; and repairing the explicit triangular mesh model based on the updated Jacobian matrix deformation field to obtain the target railway 3D model for the reconstruction of the target heavy-haul railway.

[0104] The explicit triangular mesh model is an explicit 3D geometric model composed of multiple vertices and triangular faces, used to directly represent the surface geometry of the target heavy-haul railway scene. The Jacobian matrix deformation field is a parametric deformation field used to describe the local geometric deformation of each triangular facet in the explicit triangular mesh model.

[0105] For example, such as Figure 6 As shown, the server designs a self-moving cube algorithm for railway scenes based on a denoised 3D scene reconstruction model. Using this algorithm, isosurfaces are extracted under a preset density threshold, generating a set containing vertices. With the set of face pieces Explicit triangular mesh model .

[0106] The specific formula for isosurface extraction is as follows:

[0107]

[0108] in, In three-dimensional space coordinates, The volume density value predicted by the implicit network. This is the preset threshold for the surface density of railway facilities.

[0109] Next, a parameterized representation of local deformation based on the Jacobian matrix is ​​constructed for the explicit triangular mesh model. Each triangular facet in Assign a 3×3 Jacobian transformation matrix. The Jacobian transformation matrix is ​​used to characterize the local rotation and stretching of the corresponding facet during the deformation process. The set of Jacobian matrices for all triangular facets... These constitute the potential optimization variables driving the explicit triangular mesh deformation.

[0110] Construct an optimization loop mechanism (i.e., Jacobian matrix optimization) that includes a differentiable renderer to deform the explicit triangular mesh model. A differentiable renderer is input to generate a sequence of 2D projected images from multiple randomly sampled viewpoints. Subsequently, a semantic feature encoder extracts the visual feature vectors from the 2D projected images and calculates the cosine similarity between the 2D projected images and the feature vectors of the target text instructions, constructing a multi-view semantic manifold matching loss function. This forces the mesh to semantically conform to textual descriptions such as "repairing sleepers" or "correcting roadbed" in the deformed projected view.

[0111] loss function Specifically:

[0112]

[0113] in Indicates perspective Differentiable rendering results and They are an image encoder and a text encoder, respectively. This is a repair command.

[0114] Optionally, to prevent non-physical distortions, self-intersections, or artifacts in the mesh under purely semantic-driven conditions, a local geometric topology rigidity conservation loss function is designed. This constraint can consist of two parts: first, a rigid preservation constraint, and second, a penalty Jacobian matrix. The degree of deviation from the orthogonal rotation matrix ensures that local deformations remain as rigid as possible; secondly, the neighborhood smoothing constraint penalizes the difference in the Jacobian matrix between adjacent triangular faces, ensuring the global continuity and smoothness of the overall deformation.

[0115] loss function Specifically:

[0116]

[0117] in, It is the identity matrix. For a set of adjacent facets, To smooth out the weights, It is the Frobenius norm.

[0118] Subsequently, the server uses a text encoder to convert the text repair instructions into high-dimensional semantic feature vectors. semantic feature vector The input is fed into the optimization loop, and under the premise of a fixed mesh topology, the Jacobian matrix set is used. To optimize the variables, iterative operations based on gradient descent are performed. During this process, a multi-view semantic manifold matching loss function is used. With local geometric topological rigidity conservation loss function The weighted sum of the values ​​allows the optimized Jacobian matrix to accurately represent the local geometric deformation trends that conform to the text description, while maintaining the rigidity of the local structure.

[0119] The objective function for the optimization process is as follows:

[0120]

[0121] in For the optimized most elegant comparable matrix field, For regularization weights.

[0122] The most elegant comparable matrix field capable of characterizing the deformation of the target is obtained. Subsequently, a global mesh vertex smooth displacement energy equation based on local Jacobian deformation integral was designed and constructed to find a new set of mesh vertex coordinates. This ensures that the local deformation gradient defined by the new vertex approximates the Jacobian matrix as closely as possible. The sparse linear system is solved using a Poisson equation solver, which transforms the discrete, patch-based local rotation and stretching information into a continuous, vertex-based global smooth displacement field.

[0123] The specific formula is as follows:

[0124]

[0125] in, Triangular facet area, The deformation gradient is derived from the vertex coordinates. This is the optimized Jacobian matrix.

[0126] Using the updated vertex coordinates obtained from the solution Replace the vertex data of the original mesh, repair the geometry of the explicit triangular mesh model, and obtain the target railway 3D model reconstructed for the target heavy-haul railway.

[0127] In this embodiment, the Jacobian matrix deformation field is updated based on an explicit triangular mesh model. The Jacobian matrix is ​​used to represent the local rotation and local stretching of triangular facets, allowing the 3D geometric changes to be repaired to be described by local deformation parameters. The Jacobian matrix deformation field is then optimized to match the local geometric deformation with the target repair requirements, enabling targeted adjustments to incomplete or erroneous geometric structures in the railway 3D model. Finally, the updated Jacobian matrix deformation field is used to repair the explicit triangular mesh model, converting local Jacobian deformation into global mesh vertex displacements. This allows local geometric adjustments to continuously act on the overall 3D mesh, reducing discontinuous deformations during local repair and resulting in a target railway 3D model with repaired geometry. Furthermore, since the explicit triangular mesh model originates from the aforementioned denoised 3D scene reconstruction model, the repaired model can inherit the denoised high-fidelity surface texture, achieving synergy between geometric repair and surface texture fidelity.

[0128] Optionally, such as Figure 7 As shown, the method for reconstructing the three-dimensional model of a heavy-haul railway provided in this application can also be implemented in the following ways:

[0129] First, data acquisition and semantic foundation construction are performed. Raw images are acquired using drones / inspection vehicles, and camera pose parameters are calculated. This outputs a sparse image sequence and initial scene metadata. Simultaneously, a semantic feature encoder consisting of an image encoder and a text encoder is constructed. After calculating loss optimization parameters, a frozen semantic feature encoder is output. Second, implicit field reconstruction and intelligent texture restoration are performed. First, sparse viewpoint NeRF initialization is performed. After implicit neural radiation field, volume rendering, and loss optimization parameters, an initial implicit field model of the railway scene with structural integrity is output. Then, a command-driven diffusion model is used with image conditions and text commands as input and output to restore the image, and the process proceeds iteratively. The process involves a data set update and scene optimization loop, consisting of NeRF rendering, diffusion model editing (removing weeds / enhancing textures), and hybrid dataset updates, iteratively training until convergence to output a high-quality implicit field model. Finally, explicit geometric refinement is performed. This involves first meshing, then using a self-moving cube algorithm for railway scenes to output an explicit triangular mesh model. Next, the Jacobian matrix field is constructed and optimized, sequentially building the Jacobian matrix field, performing differentiable rendering, calculating loss and optimizing parameters, outputting the most elegant comparable matrix field that characterizes the target deformation. Finally, geometric orientation optimization is performed, using Poisson equation solutions and vertex position updates to output a high-precision 3D reconstruction model of the railway after the optimization.

[0130] The initial scene metadata can be obtained through the following steps: The server uses a low-altitude drone equipped with a high-resolution camera and a track inspection vehicle to acquire image data along the planned heavy-haul railway line. The acquisition range of the drone and the track inspection vehicle covers key facilities such as roadbed slopes, rails, sleepers, fasteners, and catenary suspension devices. During the acquisition process, various shooting angles such as orthogonal, oblique, and eye-level are used to acquire multi-view image sequences containing information on different lighting conditions and depth of field, while ensuring that the forward overlap rate of adjacent images is greater than 60 degrees. The multi-view image sequence was calculated using a railway feature geometric constraint solving algorithm. The camera intrinsic matrix, camera extrinsic parameters, and sparse point cloud. Specifically, for the image set... SIFT (Scale-Invariant Feature Transform) feature extraction is performed to obtain key points. Simultaneously, an epipolar geometric constraint function adapted to the train vibration environment is constructed. This function is used to determine the geometric consistency of feature point pairs, eliminating mismatched points caused by train vibration or dynamic environmental changes, and outputting a set of feature matching pairs that satisfy the geometric constraints.

[0131] The specific epipolar geometric constraint function for adapting to train vibration environment is as follows:

[0132]

[0133] in, and These are the homogeneous coordinate vectors of corresponding feature points in two adjacent images. It is the basic matrix.

[0134] Based on the obtained feature matching pair set, two high-quality matching images are selected as seed frames for initialization, generating an initial 3D point cloud. For each subsequent added image... By utilizing the correspondence between its 2D feature points and existing 3D spatial points and the criterion of minimizing reprojection error in the local coordinate system of heavy-haul railway, the camera extrinsic parameters of the current camera in the railway coordinate system are calculated. Compared with the initial sparse point cloud.

[0135] The specific steps for minimizing the projection error in the local coordinate system of heavy-haul railways are as follows:

[0136]

[0137] in, The coordinates of the 2D observation point on the current image. These are the corresponding 3D spatial point coordinates. For camera projection function, This is the camera intrinsic parameter matrix.

[0138] Based on the calculated initial camera extrinsic parameters and initial sparse point cloud, a global optimization function is established for long-distance, narrow scenes to address the cumulative errors and trajectory drift that may occur during long-distance railway data acquisition. This function performs joint adjustment on the poses of all cameras and the coordinates of 3D points, resulting in accurate extrinsic parameters and a refined sparse point cloud for the scene.

[0139] Global optimization function for long-distance, narrow scenes Specifically as follows:

[0140]

[0141] in, For the total number of cameras, The total number of points in three-dimensional space. For visibility indicator variables (when the first) The camera observed the first... The value is 1 when there are a number of spatial points, otherwise it is 0. These are the actual observed coordinates of the feature points on the image plane. For including camera intrinsics , No. Frame extraparameters and spatial point coordinates The projection function.

[0142] Finally, the camera intrinsic parameter matrix is ​​obtained. Camera external parameters ( ) and sparse point clouds used to help define the geometric boundaries of a scene. Based on the geometric distribution features of sparse point clouds, the centroid and scaling factor of the bounding box of the scene are calculated. Centered translation and isotropic scaling are performed on the camera pose data and sparse point clouds to map them into a unit spherical space. Finally, the normalized data is divided into training, validation and test sets to obtain an initial scene metadata file containing standardized parameters.

[0143] The formulas for calculating the centroid and scaling factor are as follows:

[0144]

[0145]

[0146] in, and These represent obtaining the minimum and maximum values ​​of the point cloud on the three coordinate axes, respectively. It is a sparse dot cloud.

[0147] The specific formulas for centralized translation and isotropic scaling are as follows:

[0148]

[0149] in Let be any point in space.

[0150] In summary, the above steps first design and construct a semantically constrained implicit 3D scene reconstruction network framework adapted to the sparse observation perspective of heavy-haul railways, obtaining an initial implicit 3D model of the railway scene with structural integrity (initial 3D scene reconstruction model) under the sparse field acquisition conditions of heavy-haul railways. Subsequently, by fusing a generative diffusion model that integrates textual instructions and image conditions for unstructured environmental noise of heavy-haul railways, automated stripping and texture standardization of unstructured environmental noise such as weeds, rain, and fog are achieved. Finally, by constructing an explicit geometric repair mechanism (optimized loop mechanism) based on the Jacobian matrix field for heavy-haul railways, abstract natural language instructions are transformed into precise physical geometric deformations, quickly repairing defects and errors in the 3D model. Through the synergistic effect of these steps, a geometrically complete and high-fidelity 3D model of heavy-haul railways can be output, effectively solving the problems of limited data acquisition, difficulty in removing environmental interference, and reliance on manual geometric repair in traditional methods, providing solid technical support for the digital operation and maintenance and green low-carbon management of railways.

[0151] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0152] Based on the same inventive concept, this application also provides a device for reconstructing a three-dimensional model of a heavy-haul railway for implementing the above-described method for reconstructing a three-dimensional model of a heavy-haul railway. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations in one or more embodiments of the device for reconstructing a three-dimensional model of a heavy-haul railway provided below can be found in the limitations of the method for reconstructing a three-dimensional model of a heavy-haul railway described above, and will not be repeated here.

[0153] In one exemplary embodiment, such as Figure 8As shown, a reconstruction device for a three-dimensional model of a heavy-haul railway is provided, comprising: a data acquisition module 10, an initial modeling module 20, a reverse denoising module 30, and a repair output module 40, wherein:

[0154] The data acquisition module 10 is used to acquire multi-view images of the target heavy-haul railway and the initial scene metadata corresponding to the multi-view images. The initial scene metadata is calculated based on the multi-view images through a preset calculation algorithm, including sparse point clouds, camera intrinsic parameters and camera extrinsic parameters.

[0155] The initial modeling module 20 is used to train the 3D scene reconstruction model based on multi-view images, initial scene metadata and preset prior constraints for the 3D scene reconstruction model to be trained, so as to obtain the trained initial 3D scene reconstruction model.

[0156] The reverse denoising module 30 is used to obtain the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model, and update the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain the denoised 3D scene reconstruction model.

[0157] The repair output module 40 is used to repair the denoised 3D scene reconstruction model based on the explicit triangular mesh model determined by the denoised 3D scene reconstruction model and the pre-constructed Jacobian matrix deformation field, thereby obtaining the target railway 3D model reconstructed for the target heavy-haul railway.

[0158] In an exemplary embodiment, the initial modeling module 20 is further configured to obtain a virtual observation viewpoint different from the acquisition viewpoint corresponding to the multi-view image based on the camera extrinsic parameters included in the initial scene metadata; obtain the corresponding real viewpoint rendering map and virtual viewpoint rendering map under the acquisition viewpoint and virtual observation viewpoint corresponding to the multi-view image, respectively; and train the 3D scene reconstruction model to be trained based on the real viewpoint rendering map, virtual viewpoint rendering map, multi-view image, and preset prior constraints for the 3D scene reconstruction model to be trained, thereby obtaining the trained initial 3D scene reconstruction model.

[0159] In an exemplary embodiment, the initial modeling module 20 is further configured to determine the pixel color difference between the real-view rendering image and the multi-view image based on the real-view rendering image, the multi-view image, and the photometric reconstruction constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, and obtain the corresponding photometric loss; determine the semantic difference between the image features of the virtual-view rendering image and the multi-view image based on the virtual-view rendering image, the multi-view image, and the semantic topological invariance constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, and obtain the corresponding semantic loss; and train the 3D scene reconstruction model to be trained based on the photometric loss and the semantic loss to obtain the trained initial 3D scene reconstruction model.

[0160] In an exemplary embodiment, the reverse denoising module 30 is further configured to obtain a scene rendering image from a preset viewpoint from the initial 3D scene reconstruction model; and obtain a pseudo-real value image corresponding to the scene rendering image based on the scene rendering image.

[0161] In an exemplary embodiment, the reverse denoising module 30 is further configured to replace the image in the multi-view image corresponding to the viewpoint of the pseudo-true image based on the pseudo-true image, to obtain the updated viewpoint image corresponding to the multi-view image; and update the initial three-dimensional scene reconstruction model based on the updated viewpoint image to obtain the denoised three-dimensional scene reconstruction model.

[0162] In an exemplary embodiment, the repair output module 40 is further configured to determine the explicit triangular mesh model corresponding to the denoised 3D scene reconstruction model by using a preset isosurface extraction algorithm; update the pre-constructed Jacobian matrix deformation field based on the explicit triangular mesh model to obtain the updated Jacobian matrix deformation field; and repair the explicit triangular mesh model based on the updated Jacobian matrix deformation field to obtain the target railway 3D model for the reconstruction of the target heavy-haul railway.

[0163] The modules in the aforementioned reconstruction device for the three-dimensional model of heavy-haul railway can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0164] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores multi-view image data of the target heavy-haul railway. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for reconstructing a three-dimensional model of a heavy-haul railway.

[0165] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0166] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0167] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0168] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0169] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0172] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for reconstructing a three-dimensional model of a heavy-haul railway, characterized in that, The method includes: Acquire multi-view images of the target heavy-haul railway and determine the corresponding initial scene metadata based on the multi-view images; the initial scene metadata includes sparse point cloud, camera intrinsic parameters, and camera extrinsic parameters; Based on the multi-view images, the initial scene metadata, and the preset prior constraints for the 3D scene reconstruction model to be trained, the 3D scene reconstruction model to be trained is trained to obtain the initial 3D scene reconstruction model after training. Obtain the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model, and update the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain a denoised 3D scene reconstruction model. Based on the denoised 3D scene reconstruction model, an explicit triangular mesh model is determined. Based on the pre-constructed Jacobian matrix deformation field, the explicit triangular mesh model is repaired to obtain a target railway 3D model for the reconstruction of the target heavy-haul railway.

2. The method according to claim 1, characterized in that, The process of training the 3D scene reconstruction model based on the multi-view images, the initial scene metadata, and preset prior constraints for the 3D scene reconstruction model to be trained, to obtain a trained initial 3D scene reconstruction model, includes: Based on the camera extrinsic parameters included in the initial scene metadata, a virtual observation perspective different from the acquisition perspective corresponding to the multi-view image is obtained; Obtain the corresponding real-view rendering image and virtual-view rendering image under the acquisition view and the virtual observation view corresponding to the multi-view image, respectively; Based on the real-view rendering image, the virtual-view rendering image, the multi-view image, and the preset prior constraints for the 3D scene reconstruction model to be trained, the 3D scene reconstruction model to be trained is trained to obtain the initial 3D scene reconstruction model after training.

3. The method according to claim 2, characterized in that, The process involves training the 3D scene reconstruction model based on the real-view rendering image, the virtual-view rendering image, the multi-view images, and preset prior constraints for the 3D scene reconstruction model to be trained, to obtain a trained initial 3D scene reconstruction model; including: Based on the real-view rendering, the multi-view image, and the photometric reconstruction constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, the pixel color difference between the real-view rendering and the multi-view image is determined, and the corresponding photometric loss is obtained. Based on the virtual view rendering, the multi-view images, and the semantic topological invariance constraints included in the preset prior constraints for the 3D scene reconstruction model to be trained, the semantic differences between the image features of the virtual view rendering and the multi-view images are determined, and the corresponding semantic loss is obtained. Based on the photometric loss and the semantic loss, the 3D scene reconstruction model to be trained is trained to obtain the initial 3D scene reconstruction model after training.

4. The method according to claim 1, characterized in that, The step of obtaining the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model includes: Obtain the scene rendering image from the preset viewpoint from the initial 3D scene reconstruction model; Based on the scene rendering image, a pseudo-real value image corresponding to the scene rendering image is obtained.

5. The method according to claim 1, characterized in that, The step of updating the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain a denoised 3D scene reconstruction model includes: Based on the pseudo-true value image, replace the image in the multi-view image that corresponds to the view of the pseudo-true value image to obtain the updated view image corresponding to the multi-view image; Based on the updated viewpoint image, the initial 3D scene reconstruction model is updated to obtain a denoised 3D scene reconstruction model.

6. The method according to claim 1, characterized in that, The explicit triangular mesh model is determined based on the denoised 3D scene reconstruction model. Based on a pre-constructed Jacobian matrix deformation field, the explicit triangular mesh model is repaired to obtain a target railway 3D model for the reconstruction of the target heavy-haul railway, including: The explicit triangular mesh model corresponding to the denoised 3D scene reconstruction model is determined by a preset isosurface extraction algorithm. Based on the explicit triangular mesh model, the pre-constructed Jacobian matrix deformation field is updated to obtain the updated Jacobian matrix deformation field. Based on the updated Jacobian matrix deformation field, the explicit triangular mesh model is repaired to obtain a three-dimensional model of the target railway reconstructed for the target heavy-haul railway.

7. A device for reconstructing a three-dimensional model of a heavy-haul railway, characterized in that, The device includes: The data acquisition module is used to acquire multi-view images of the target heavy-haul railway and initial scene metadata corresponding to the multi-view images; the initial scene metadata is calculated based on the multi-view images using a preset calculation algorithm, including sparse point cloud, camera intrinsic parameters and camera extrinsic parameters. The initial modeling module is used to train the 3D scene reconstruction model to be trained based on the multi-view images, the initial scene metadata, and the preset prior constraints for the 3D scene reconstruction model to be trained, so as to obtain the trained initial 3D scene reconstruction model. The reverse denoising module is used to obtain the pseudo-ground value image corresponding to the rendered image of the initial 3D scene reconstruction model, and update the initial 3D scene reconstruction model based on the pseudo-ground value image to obtain the denoised 3D scene reconstruction model. The repair output module is used to repair the denoised 3D scene reconstruction model based on the explicit triangular mesh model determined by the denoised 3D scene reconstruction model and the pre-constructed Jacobian matrix deformation field, to obtain a target railway 3D model reconstructed for the target heavy-haul railway.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.