A Method and Computer System for Constructing Digital Twin Models of Roadbed Slopes Based on Multi-View Array Vision Perception

Abstract

This invention discloses a method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception. A preliminary digital twin model is created by combining slope design drawings and construction plans. Different types and numbers of cameras are set up to form a multi-view visual perception array according to the construction site conditions. Images showing changes during construction are corrected, enhanced, and denoised to extract key construction features, which are then processed using deep learning algorithms. The digital twin model of the roadbed slope is updated and refined using 3D reconstruction technology. The digital twin model is updated in real time, and structural analysis and dynamic simulation are performed to assess potential risks at different construction stages. Based on risk identification, emergency response strategies and preventative measures are formulated. The advantages of this invention are that it achieves efficient, real-time, and accurate monitoring of changes during slope construction, providing strong support for construction process monitoring, quality assurance, and risk prediction.

Description

Technical Field

[0001] This invention relates to the field of intelligent slope control, and in particular to a method and computer system for constructing a digital twin model of a roadbed slope based on multi-view array visual perception. Background Technology

[0002] Roadbed slopes are a crucial component of transportation infrastructure construction, and their deformation stability directly impacts traffic safety and the long-term performance of the project. Traditionally, quality, progress monitoring, and monitoring of changes in the site environment during roadbed slope construction rely heavily on manual methods. This is not only inefficient but also susceptible to human error, making it impossible to accurately and in real-time reflect changes at the construction site.

[0003] With the rapid development of digital and intelligent technologies, digital twin technology, as a novel approach, has been widely applied in various fields. Digital twins, through the high integration of physical entities and virtual models, simulate and monitor the dynamic changes of the physical world in real time and update the virtual model accordingly, thereby improving the management level and construction efficiency of engineering projects. In the field of roadbed slope construction, combining digital twin technology can more accurately reflect various state changes during construction, identify and predict potential risks in advance, and ensure the safety and quality of construction. However, existing digital twin models mostly rely on single-view visual perception technology, neglecting the acquisition of multi-angle and multi-level information in the complex environment of the construction site, resulting in the inability to comprehensively and accurately capture the detailed changes during construction. To achieve more efficient and comprehensive monitoring and management of the construction process, a new visual perception method is needed that can provide multi-view, multi-dimensional information, thereby supporting the construction of digital twin models for roadbed slope construction. This is the area that this application focuses on improving. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a method and computer system for constructing a digital twin model of roadbed slope based on multi-view array visual perception, so as to achieve efficient, real-time and accurate monitoring of changes in the slope construction process, and provide strong support for monitoring, quality assurance and risk prediction of the construction process.

[0005] To address the aforementioned technical problems, this invention provides a method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception. This method involves creating a preliminary digital twin model by combining slope design drawings and construction plans, and then setting up a multi-view visual perception array with different types and numbers of cameras according to the construction site conditions. Images of slope changes during construction are obtained using the multi-view visual perception array. These images are then corrected, enhanced, and denoised. Key construction features are extracted from the improved images using a deep learning algorithm. Based on the digital image feature extraction results, 3D reconstruction technology is used to update and refine the roadbed slope digital twin model, which is updated in real time. The updated digital twin model is then used for detailed structural analysis and dynamic simulation to assess potential risks at different construction stages. Based on risk identification, emergency response strategies and preventative measures are developed to address predicted potential safety issues and ensure high-quality project construction. Specifically, the method includes the following steps:

[0006] Step S1: Integrate the design drawings and construction planning documents for slope construction, including topographic and geological data, engineering layout, and planned construction schemes; use modeling tools to create a preliminary digital twin model of the slope based on the design drawings and construction plans;

[0007] Step S2: Based on the design drawings, construction plan and specific site conditions, set up different types and quantities of cameras, and plan their installation locations to fully cover all key areas, forming a multi-view visual perception array;

[0008] Choose the appropriate camera type, such as a fixed, rotating, or drone camera, install and debug the camera system to ensure that all devices can operate stably and provide high-quality image data;

[0009] The camera system is activated to acquire real-time video and images, enabling the data to be transmitted securely and efficiently to the central processing unit for subsequent processing and analysis.

[0010] Step S3: Obtain images of slope changes during construction using a multi-view visual perception array, and perform image correction, enhancement, and noise reduction processing; use deep learning algorithms to automatically identify and extract key construction features from the processed images, and match the construction feature data with preset monitoring parameters for subsequent analysis;

[0011] The collected images are subjected to geometric and color correction to eliminate distortion and improve image quality;

[0012] Image enhancement techniques are applied to process the images to clearly highlight key construction features.

[0013] Step S4: Based on the feature extraction results, update and refine the digital twin model using 3D reconstruction technology;

[0014] During the reconstruction process, ensure that the model accurately reflects the actual situation on site, including any temporary structures and changes in construction progress; continuously integrate real-time data and 3D reconstruction results into the initial digital twin model to ensure that the model always stays up-to-date and accurately reflects the dynamic changes on the construction site;

[0015] Step S5: Use the updated digital twin model to perform detailed structural analysis and dynamic simulation to assess potential problems and risks at different construction stages;

[0016] Based on the results of the model analysis, implementation recommendations are provided to the project management team, including adjusting the construction plan, optimizing resource allocation and improving safety measures, and developing emergency response strategies and preventive measures to address potential problems predicted in the analysis.

[0017] Step S6: Continuously use multi-view vision sensing arrays and other sensor devices for real-time monitoring of the construction site;

[0018] Continuously collect data to support the real-time updating and analysis of the digital twin model; repeat steps S3-S5 repeatedly with new data to continuously provide feedback to the project and ensure its safe implementation.

[0019] In step S1, all slope topographic and geological data are collected and reviewed. This information forms the basis for establishing a preliminary model. The preliminary digital twin model of the slope is established using the finite element analysis software ABAQUS based on the collected data.

[0020] In step S2, a multi-view visual perception array is constructed to achieve full coverage and dynamic monitoring of the slope construction site. The multi-view visual perception array includes various types of cameras deployed at preset positions and angles to form a spatially complementary perception network. Each camera in the array not only works independently, but also provides multi-angle and multi-spectral visual perception capabilities through data fusion and collaborative processing.

[0021] Specifically, wide-angle cameras are used to cover a large area and monitor the overall construction dynamics, deployed at a certain distance from the construction site; telephoto cameras focus on capturing details of key construction points and are deployed close to critical construction areas, with their focal length flexibly adjusted according to actual construction needs; drone cameras, as mobile nodes in the array, provide a dynamic aerial perspective, flexibly monitoring areas of slopes that are difficult to monitor from a fixed location. In addition, standard-focal-length cameras are deployed in the blank areas of the array, further improving visual perception through cross-coverage of the field of view, eliminating blind spots, and enhancing the redundancy and reliability of monitoring.

[0022] The entire multi-view vision sensing array must output video at a resolution of 1080p / 30fps or higher. Infrared cameras may be used to enhance monitoring performance in low-light conditions, depending on the situation. To ensure efficient data transmission, cameras closer to the construction area transmit data wirelessly, while cameras further away are connected to the data center via wired connections, enabling real-time and reliable transmission and processing of the multi-view vision sensing array's data.

[0023] By constructing a multi-view visual perception array, we can achieve all-round, blind-spot-free visual perception of slope construction. Furthermore, by integrating various perception data, we can improve the accuracy and operability of the digital twin model.

[0024] Multi-view visual perception array C = {C1, C2, ..., C n}, through a reasonable arrangement of cameras, comprehensive monitoring of the construction site is achieved. The position of each camera (x i ,y i ,z i The arrangement of the field of view θi and the field of view θi must satisfy specific geometric and field of view coverage constraints in order to achieve the optimal perception effect;

[0025] For each camera C i Its coverage area A(C i ) is represented as:

[0026] Where: r i θ is the distance from the camera to the monitored area. i It's the camera's perspective;

[0027] To ensure seamless and comprehensive coverage of the entire construction site, the sum of the coverage areas of all cameras must be no less than the total area A of the construction site. total ,Right now:

[0028] According to this formula, by adjusting the position and angle θ of the camera... i This is to achieve comprehensive monitoring of the entire site. Assume the total area of ​​the site is A. total Then the total coverage area of ​​the multi-view vision perception array is not less than A. total To avoid blind spots in monitoring.

[0029] To reduce blind spots while ensuring a certain degree of redundancy, the fields of view of multi-camera systems should have appropriate overlap. Define two cameras C... i and C j The overlapping area of ​​the field of view O(C) i C j )for:

[0030] O(C i Cj )=A(C i )∩A(C j ),

[0031] To ensure redundancy, the total area of ​​the overlapping regions must meet certain conditions: Where α is the redundancy coefficient, which is usually taken as 0.1≤α≤0.2 to ensure the reliability of the monitoring system.

[0032] In step S3, functions from the OpenCV library are used to adjust the image based on known distortion coefficients and camera parameters, thereby restoring the original viewpoint and proportions of the image; Adobe Photoshop is used to improve the color balance of the image and enhance image quality; color balance techniques are used to adjust the white balance of the image, or histogram equalization is used to improve the contrast and detail visibility of the image; machine learning-based methods, such as autoencoders, are used to reduce noise while preserving important image features; in the extraction of construction features, a pre-trained CNN model, VGG-16, is selected, which has been trained on a large amount of image data and effectively extracts image features; transfer learning is performed using the pre-trained model by replacing the last few layers of the network and then training these layers on a specific dataset with construction features to adapt the model to a specific task; and the MiDaS model is loaded using OpenCV and PyTorch for depth estimation to obtain the slope excavation depth.

[0033] Step S4 involves synchronizing the changes in the 3D model to the finite element analysis software in real time to achieve real-time updates of the digital twin model.

[0034] In step S5, the updated digital twin model is analyzed in finite element analysis software. If the slope instability is reduced to a certain threshold due to construction, the construction method or sequence is adjusted to mitigate the risk, or specific safety measures such as strengthening temporary supports are proposed to ensure the normal progress of construction. Based on the real-time updated digital twin model, corresponding action plans are established for crisis situations, such as soil slippage and severe weather events.

[0035] The present invention also provides a computer system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception.

[0036] The beneficial effects of this invention are as follows:

[0037] 1) This invention uses a digital twin model to analyze and monitor the real-time construction progress of slopes. Compared with other detection and construction methods, it can monitor slopes more timely and effectively, and provide timely feedback on the stability of the slope under the current construction status at the construction site. It can also avoid the obstruction of slope construction due to special circumstances during the construction process.

[0038] 2) This invention uses a multi-view visual perception array to monitor slopes. Compared with a monocular visual perception system, the multi-view visual perception array captures images from different angles through multiple cameras, which not only avoids visual blind spots but also provides more accurate image data, which is beneficial for the generation of digital twin models. Furthermore, the multiple cameras have visual overlap, forming redundancy. Even if some cameras fail, the system can still operate normally, ensuring the stability of monitoring. Attached Figure Description

[0039] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0040] Figure 1 This is a flowchart of a specific embodiment of the present invention;

[0041] Figure 2 This is a preliminary digital twin model diagram of a specific embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the camera setup at a slope construction site according to a specific embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the corrected slope according to a specific embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of depth extraction in a specific embodiment of the present invention;

[0045] Figures 6(a) and (b) show digital twin models of specific embodiments of the present invention;

[0046] Figure 7 The safety factor calculation results are for a specific embodiment of the present invention;

[0047] 1—Wide-angle camera; 2—Telephoto camera;

[0048] 3—Unmanned Aerial Vehicle (UAV) patrol team; 4—Slope;

[0049] 5 — Highway; 6 — Section used for extracting slope data. Detailed Implementation

[0050] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] like Figure 1 As shown, this invention provides a method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception. A preliminary digital twin model is created by combining slope design drawings and construction plans. Different types and numbers of cameras are set up to form a multi-view visual perception array according to the construction site conditions. Images of slope changes during construction are obtained through the multi-view visual perception array. These images are corrected, enhanced, and denoised. Key construction features are extracted from the improved images using deep learning algorithms. Based on the digital image feature extraction results, 3D reconstruction technology is used to update and refine the roadbed slope digital twin model, which is updated in real time. The updated digital twin model is used for detailed structural analysis and dynamic simulation to assess potential risks at different construction stages. Based on risk identification, emergency response strategies and preventative measures are formulated to address the predicted potential safety issues. A specific example is taken from the slope construction of a highway in Guangxi.

[0052] Step S1: Integrate the design drawings and construction planning documents for the slope construction of a highway in Guangxi, and use the finite element analysis software ABAQUS to create a preliminary digital twin model of the slope, such as... Figure 2 .

[0053] Step S2: Based on the specific conditions of a highway slope, select appropriate cameras. In this embodiment, a wide-angle camera 1, a telephoto camera 2, and a drone patrol group 3 are selected to form a multi-view visual array for perception, achieving full coverage and dynamic monitoring of the slope construction site. Figure 3 As shown. All cameras output at 1080p / 30fps or higher. Wide-angle camera 1 is located opposite slope 4, and telephoto camera 2 is located on both sides of slope 4 to supplement the shooting. Drone 3 patrol team conducts regular patrols to supplement the slope data from above.

[0054] The camera parameters involved are as follows:

[0055] Wide-angle camera (C1): Monitoring range radius r1 = 50m, viewing angle θ1 = 120°;

[0056] Telephoto cameras (C2, C3): Monitoring range radius r2=r3=30m, viewing angle θ2=θ3=20°;

[0057] Drone camera (C4): Monitoring range radius r4 = 70m, viewing angle θ4 = 45°;

[0058] Coverage area calculation:

[0059] 1) Calculate the coverage area A(C) of each camera. i ): A(C i)=1 / 2×r1 2 θ i

[0060] Wide-angle camera C1: A(C1) = 1 / 2 × 50 2 ×2π / 3≈2617.99m 2

[0061] For telephoto cameras C2 and C3: A(C2) = A(C3) = 1 / 2 × 30 2 ×π / 9≈157.08m 2

[0062] Drone camera C4: A(C4) = 1 / 2 × 70 2 ×π / 4≈1923.07m 2 ;

[0063] 2) Calculation of total coverage area:

[0064] 3) Overlapping Area Calculation: The redundancy coefficient α is set to 0.1. There is some overlap between the cameras, and the average area of ​​the overlapping region is 10% of the coverage area: O(C i C j )=0.1×A(C i )

[0065] The total area of ​​the overlapping regions is:

[0066]

[0067] 4) Check for redundancy conditions:

[0068]

[0069] The construction site area, based on on-site inspection, is 390.71 square meters (≥ 0.10 × 3000 = 300 square meters), totaling 3000 square meters. 2 ;

[0070] The overlap area exceeds the redundancy requirement, thus meeting the requirements.

[0071] Step S3: Use the Cv2 function from the OpenCV library to adjust the image based on the distortion coefficients and camera parameters. (This embodiment...) To enhance image quality, the pre-trained CNN model VGG-16 was used to extract construction features, and the MiDaS model was loaded using OpenCV and PyTorch for depth estimation to obtain the slope excavation depth. The corrected slope diagram in this embodiment is shown below. Figure 4 As shown, the extracted slope depth is as follows: Figure 5 As shown, Figure 5 The results show that the excavated part of the slope is about 5m lower than the unexcavated part.

[0072] Step S4: Synchronize the 5m excavation depth of the slope to the preliminary digital twin model of the slope, update the preliminary digital twin model of the slope, and analyze the deformation and plastic strain distribution of the slope, such as... Figure 6a and Figure 6b As shown, Figure 6a This is a diagram illustrating the cumulative displacement. Figure 6b This is a diagram showing the distribution of plastic strain.

[0073] Step S5: Perform strength reduction calculations on the digital twin model of the slope using the finite element analysis software ABAQUS, analyze the safety factor of the slope after construction, and based on the extracted slope safety factor diagram, from... Figure 7 The results show that the safety factor of the slope is greater than 1.8, which is much greater than the standard requirement of 1.3-1.5, indicating that the slope is relatively safe during construction.

[0074] The present invention also provides a computer system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception.

[0075] This invention proposes a method for constructing a digital twin model of roadbed slopes based on multi-view array visual perception. When building the digital twin model, it combines multi-view monitoring of the slope construction process to create a more accurate digital twin model. The multi-view visual perception array avoids the shortcomings of traditional monocular visual perception, providing accurate and real-time images of slope construction progress, greatly assisting in the development of the digital twin model. The combination of the multi-view visual perception array and the digital twin model of the slope can provide a new direction for intelligent slope construction, avoiding special situations that may occur during slope construction and ensuring the normal progress of slope construction.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception, comprising the following specific steps: Step S1: Integrate the design drawings and construction planning documents for slope construction, including topographic and geological data, engineering layout, and planned construction schemes; use modeling tools to create a preliminary digital twin model of the slope based on the design drawings and construction plans; Step S2: Based on the design drawings, construction plan and specific site conditions, set up different types and quantities of cameras, and plan their installation locations to fully cover all key areas, forming a multi-view visual perception array; Step S3: Obtain images of slope changes during construction using a multi-view visual perception array, and perform image correction, enhancement, and noise reduction processing; use deep learning algorithms to automatically identify and extract key construction features from the processed images, and match the construction feature data with preset monitoring parameters for subsequent analysis; Step S4: Based on the feature extraction results, update and refine the digital twin model using 3D reconstruction technology; Step S5: Use the updated digital twin model to conduct detailed structural analysis and dynamic simulation, assess the problems and risks that may occur at different construction stages, and, in conjunction with risk identification, formulate emergency response strategies and preventive measures to address the potential safety issues predicted in the analysis. Step S6: Continuously use a multi-view vision sensing array for real-time monitoring of the construction site.

2. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 1, characterized in that: In step S2, the multi-view visual perception array includes various types of cameras deployed at preset positions and angles to form a spatially complementary perception network. Each camera in the array not only works independently but also provides multi-angle and multi-spectral visual perception capabilities through data fusion and collaborative processing.

3. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 2, characterized in that: The cameras include wide-angle cameras, telephoto cameras, drone cameras, and standard focal length cameras. The wide-angle cameras are deployed at a certain distance from the construction site to cover a large area and monitor the overall construction dynamics. The telephoto cameras are deployed close to key construction areas, focusing on capturing details of key construction points, and their focal length is adjusted according to actual construction needs. The drone cameras provide a dynamic aerial view. The standard focal length cameras are deployed in the blank areas of the array, improving visual perception through cross-coverage of the field of view.

4. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 2 or 3, characterized in that: The multi-view vision perception array Arrange the cameras, the position of each camera and field of view The arrangement must meet specific geometric and field-of-view coverage constraints; For each camera Its coverage area Represented as: (1); in: It is the distance from the camera to the monitored area. It's the camera's perspective; To ensure seamless and comprehensive coverage of the entire construction site, the sum of the coverage areas of all cameras must be no less than the total area of ​​the construction site. ,Right now: (2); According to formula (1), by adjusting the position and angle of the camera... To achieve comprehensive monitoring of the entire site; the total area of ​​the site is Then the total coverage area of ​​the multi-view vision perception array is not less than .

5. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 4, characterized in that: The fields of view of multiple cameras should have appropriate overlap: Define two cameras and Overlapping areas of the field of view for: , To ensure redundancy, the total area of ​​the overlapping regions must meet certain conditions: ; in The redundancy coefficient is taken as... .

6. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 1, characterized in that: Step S3, the image correction, involves performing geometric and color corrections; and applying image enhancement techniques to process the image to clearly highlight key construction features in the image.

7. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 1, characterized in that: In step S3, a deep learning algorithm is used. The pre-trained CNN model VGG-16 is selected for transfer learning. The last few layers of the network are replaced, and these layers are trained on a specific dataset with construction characteristics to adapt the model to the specific task. The MiDaS model is loaded using OpenCV and PyTorch to perform depth estimation and obtain the slope excavation depth.

8. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 1, characterized in that: Step S4 involves synchronizing the changes in the 3D model to the finite element analysis software in real time to achieve real-time updates of the digital twin model.

9. The method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception according to claim 1, characterized in that: The updated digital twin model was analyzed in finite element analysis software. It was found that the slope instability was reduced to a certain threshold due to construction. The construction methods or sequence were adjusted to mitigate the risks, or specific safety measures were proposed to ensure the normal progress of construction.

10. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method for constructing a digital twin model of a roadbed slope based on multi-view array visual perception as described in any one of claims 1-9.

Images