Roadbed slope digital twinborn model construction method based on multi-view array visual perception and computer system

Abstract

The invention discloses a roadbed slope digital twinborn model construction method based on multi-view array visual perception, and the method comprises the steps: making a preliminary digital twinborn model through combining a slope design drawing and a construction plan, and setting different types and numbers of cameras according to the conditions of a construction site to form a multi-view visual perception array; performing correction, enhancement and noise reduction processing on a change condition picture in the construction process, extracting key construction features, and using a deep learning algorithm; the three-dimensional reconstruction technology is used for updating and refining the roadbed slope digital twinborn model, the digital twinborn model is updated in real time, structural analysis and dynamic simulation are conducted on the digital twinborn model, risks possibly occurring in different construction stages are evaluated, and emergency response strategies and prevention measures are formulated in combination with risk recognition. The method has the advantages that the change in the slope construction process is efficiently and accurately monitored in real time, and powerful support is provided for monitoring, quality assurance and risk prediction of the construction process.

Description

Technical Field

[0001] The present invention relates to the field of intelligent slope control, and in particular to a method and a computer system for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception. Background Art

[0002] Roadbed slopes are an important part of transportation infrastructure construction, and their deformation stability is directly related to traffic safety and the long-term performance of the project. In the traditional roadbed slope construction process, quality and progress monitoring and on-site environmental changes often rely on manual monitoring, which is not only inefficient, but also easily restricted by human factors, resulting in the inability to reflect changes on the construction site in real time and accurately.

[0003] With the rapid development of digital and intelligent technology, digital twin technology, as a new means, has been widely used in many fields. Digital twins simulate and monitor the dynamic changes of the physical world in real time and implement updates of virtual models through the high integration of physical entities and virtual models, thereby improving the management level and construction efficiency of engineering projects. In the field of roadbed slope construction, the combination of digital twin technology can more accurately reflect the various state changes in the construction process, identify and predict potential risks in advance, and ensure the safety and quality of construction. However, the existing digital twin models mostly rely on single-perspective visual perception technology, ignoring the acquisition of multi-angle and multi-level information in the complex environment of the construction site, resulting in the inability to fully and accurately capture the details of the construction process. Changes. In order to achieve more efficient and comprehensive construction process monitoring and management, a new type of visual perception method is needed that can provide multi-perspective and multi-dimensional information, thereby providing support for the construction of digital twin models for roadbed slope construction. This is where this application needs to focus on improvement. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and a computer system for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception, so as to achieve efficient, real-time and accurate monitoring of changes in the slope during construction, and provide strong support for monitoring, quality assurance and risk prediction of the construction process.

[0005] In order to solve the above technical problems, the present invention provides a method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception. A preliminary digital twin model is produced by combining the slope design drawing and the construction plan, and different types and numbers of cameras are set according to the conditions of the construction site to form a multi-eye visual perception array; a picture of the change of the slope during the construction process is obtained through the multi-eye visual perception array, and the picture is corrected, enhanced and denoised, and the key construction features are extracted from the improved image, and a deep learning algorithm is used; according to the digital image feature extraction results, the three-dimensional reconstruction technology is used to update and refine the digital twin model of the roadbed slope, and the digital twin model is updated in real time; the updated digital twin model is used to perform detailed structural analysis and dynamic simulation, evaluate the risks that may occur in different construction stages, and combine risk identification to formulate emergency response strategies and preventive measures to deal with potential safety issues predicted by analysis, so as to ensure the high-quality development of engineering construction. Specifically, the following steps are included:

[0006] Step S1: Integrate the design drawings and construction planning documents of the slope construction, including topographic and geological data, engineering layout, and predetermined construction plan; use modeling tools to create a preliminary digital twin model of the slope based on the design drawings and construction plan;

[0007] Step S2: According to the design drawings, construction plan and specific conditions on site, different types and numbers of cameras are set up, and their installation locations are planned to fully cover all key areas to form a multi-eye visual perception array;

[0008] Select the appropriate camera type, such as fixed, rotating or drone cameras, install and debug the camera system, and ensure that all equipment can operate stably and provide high-quality image data;

[0009] Start the camera system to collect real-time video and images, and transmit the data to the central processing unit in a safe and efficient manner for subsequent processing and analysis;

[0010] Step S3: Obtain an image of the slope changes during the construction process through a multi-eye visual perception array, and perform correction, enhancement and noise reduction on the image; use a deep learning algorithm to automatically identify and extract key construction features from the processed image, and match the construction feature data with preset monitoring parameters for subsequent analysis;

[0011] Perform geometric correction and color correction on the collected images to eliminate distortion and improve image quality;

[0012] Apply image enhancement technology to process the image to clearly highlight the key construction features in the image;

[0013] Step S4: Based on the feature extraction results, the digital twin model is updated and refined using three-dimensional reconstruction technology;

[0014] During the reconstruction process, ensure that the model accurately reflects the actual situation on site, including any temporary structures and changes in the construction progress; continuously integrate real-time data and 3D reconstruction results into the preliminary digital twin model to ensure that the model is always up to date and accurately reflects the dynamic changes on the construction site;

[0015] Step S5: Use the updated digital twin model to conduct detailed structural analysis and dynamic simulation to assess possible problems and risks at different construction stages;

[0016] Provide implementation recommendations to the project management team based on the results of the model analysis, including adjusting the construction plan, optimizing resource allocation, improving safety measures, and developing emergency response strategies and preventive measures to address potential problems predicted by the analysis;

[0017] Step S6: Continuously use the multi-eye visual perception array and other sensor devices to monitor the construction site in real time;

[0018] Continuously collect data to support real-time updating and analysis of the digital twin model; repeat steps S3-S5 based on the new data obtained, and provide continuous feedback to the project to ensure its safe implementation.

[0019] In step S1, all the topographic and geological data of the slope are collected and reviewed. This information is the basis for establishing a preliminary model. The finite element analysis software ABAQUS is used to establish a preliminary digital twin model of the slope based on the collected data.

[0020] In step S2, full coverage and dynamic monitoring of the slope construction site are achieved by constructing a multi-eye visual perception array, wherein the multi-eye visual perception array includes multiple types of cameras deployed at preset positions and angles to form a spatially complementary perception network; each camera in the array not only works independently, but also provides multi-angle and multi-spectral visual perception capabilities through data fusion and collaborative processing;

[0021] Specifically, wide-angle cameras are used to cover a large area, monitor the overall construction dynamics, and are deployed at a certain distance from the construction site; telephoto cameras focus on capturing details of key construction points, are deployed near key construction areas, and their focal lengths are flexibly adjusted according to actual construction needs; drone cameras, as mobile nodes in the array, provide a dynamic perspective from the air, and flexibly monitor areas on the slope that are difficult to monitor in a fixed manner. In addition, ordinary focal length cameras are deployed in blank areas of the array, and through cross-coverage of the field of view, the visual perception effect is further improved, blind spots in monitoring are eliminated, and the redundancy and reliability of monitoring are improved;

[0022] The camera output video specification of the entire multi-eye visual perception array must reach 1080p / 30fps or above, and infrared cameras can be used to enhance the monitoring effect under low-light conditions. In order to ensure the efficiency of data transmission, cameras closer to the construction area transmit data via wireless networks, and cameras farther away use wired connections to the data center, so that the perception data of the multi-eye visual perception array can be transmitted and processed in real time and reliably;

[0023] By constructing a multi-eye visual perception array, all-round, no-blind-angle visual perception of slope construction is achieved, and by integrating multiple perception data, the accuracy and operability of the digital twin model are improved.

[0024] Multi-eye visual perception array C = {C1, C2, ..., C n}, through reasonable camera arrangement, the construction site is monitored in all directions. The position of each camera (x i ,y i ,z i ) and the arrangement of the field of view θi need to meet specific geometric and field of view coverage constraints to achieve the best perception effect;

[0025] For each camera C i , which covers area A(C i ) is expressed as:

[0026] Where: r i is the distance from the camera to the monitoring area, θ i is the camera's viewing angle;

[0027] In order to achieve seamless and comprehensive coverage of the entire construction site, the sum of the coverage areas of all cameras is required to be no less than the total area A of the construction site. total ,Right now:

[0028] According to this formula, by adjusting the position and viewing angle θ of the camera i To achieve comprehensive monitoring of the entire site. Assume that the total area of ​​the site is A total , then the total coverage area of ​​the multi-view visual perception array is not less than A total , to avoid monitoring blind spots.

[0029] In order to reduce blind spots in the field of view and ensure a certain degree of redundancy, the fields of view of multiple cameras should overlap appropriately. Define two cameras C i and C j The field of view overlap area O(C i ,C j )for:

[0030] O(C i ,Cj )=A(C i )∩A(C j ),

[0031] In order to ensure redundancy, the total area of ​​the overlapping region is required to meet certain conditions: Where α is the redundancy coefficient, which is usually taken as 0.1≤α≤0.2 to ensure the reliability of the monitoring system.

[0032] In step S3, the image is adjusted according to the known distortion coefficient and camera parameters using functions in the OpenCV library, thereby restoring the original perspective and proportion of the image; Adobe Photoshop is used to improve the color balance of the image and enhance the image quality; the white balance of the image is adjusted using color balance technology, or the contrast and detail visibility of the image are improved using histogram equalization; a machine learning-based method such as an autoencoder is used to reduce noise while retaining important features of the image; in the extraction of construction features, a pre-trained CNN model VGG-16 is selected, which has been trained on a large amount of image data and effectively extracts image features; the pre-trained model is used for transfer learning, by replacing the last few layers of the network and then training these layers on a specific data set with construction features to adapt the model to a specific task; OpenCV and PyTorch are used to load the MiDaS model for depth estimation to obtain the slope excavation depth.

[0033] In step S4, the changes of the three-dimensional model are synchronized to the finite element analysis software in real time to achieve real-time updating of the digital twin model.

[0034] In step S5, the updated digital twin model is analyzed in finite element analysis software. If the slope instability caused by construction is reduced to a certain threshold, the construction method or sequence is adjusted to reduce the risk, or specific safety measures such as strengthening temporary support are proposed to ensure the normal progress of construction. According to the real-time updated digital twin model, an action plan is established for the corresponding crisis situation, soil sliding, severe weather events, etc.

[0035] The present invention also provides a computer system, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception.

[0036] The beneficial effects of the present invention are:

[0037] 1) The present invention adopts the method of digital twin model to analyze and monitor the real-time construction progress of the slope. Compared with other detection and construction methods, it can monitor the slope more timely and effectively, and timely feedback the stability of the slope under the current construction state to the construction site, and avoid special circumstances during the construction process that may cause obstruction of the slope construction;

[0038] 2) The present invention adopts a multi-eye visual perception array to monitor the slope. Compared with the monocular visual perception system, the multi-eye visual perception array captures images from different angles through multiple cameras, which not only avoids the occurrence of visual blind spots but also can provide more accurate image data, which is conducive to the generation of digital twin models; and the multiple cameras have visual overlap to form redundancy, even if some cameras fail, the normal operation of the system can be guaranteed, ensuring the stability of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0040] Figure 1 is a flow chart of a specific embodiment of the present invention;

[0041] Figure 2 This is a preliminary digital twin model diagram of a specific embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of arranging cameras at a slope construction site according to a specific embodiment of the present invention;

[0043] Figure 4 It is a schematic diagram of the slope after correction in a specific embodiment of the present invention;

[0044] Figure 5 It is a schematic diagram of depth extraction according to a specific embodiment of the present invention;

[0045] Figure 6 (a) and (b) are digital twin models of a specific embodiment of the present invention;

[0046] Figure 7 The safety factor calculation result of the specific embodiment of the present invention;

[0047] 1—Wide-angle camera; 2—Telephoto camera;

[0048] 3—UAV cruise group; 4—slope;

[0049] 5—highway; 6—part used to extract slope data. DETAILED DESCRIPTION

[0050] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] like Figure 1 As shown, the present invention provides a method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception. A preliminary digital twin model is produced by combining the slope design drawing and the construction plan, and different types and numbers of cameras are set according to the conditions of the construction site to form a multi-eye visual perception array; a picture of the change of the slope during the construction process is obtained through the multi-eye visual perception array, and the picture is corrected, enhanced and denoised, and key construction features are extracted from the improved image, and a deep learning algorithm is used; according to the digital image feature extraction results, the three-dimensional reconstruction technology is used to update and refine the digital twin model of the roadbed slope, and the digital twin model is updated in real time; the updated digital twin model is used to perform detailed structural analysis and dynamic simulation, evaluate the risks that may occur in different construction stages, and combine risk identification to formulate emergency response strategies and preventive measures to deal with potential safety issues predicted by analysis. Take the slope construction of a highway in Guangxi as an example for specific explanation:

[0052] Step S1: Integrate the design drawings and construction planning documents of a highway slope construction in Guangxi, and use the finite element analysis software ABAQUS to create a preliminary digital twin model of the slope, such as Figure 2 .

[0053] Step S2: According to the site conditions of a highway slope, a suitable camera is selected. In this embodiment, a wide-angle camera 1, a telephoto camera 2 and a drone cruise group 3 are selected to form a multi-eye visual array for perception, so as to achieve full coverage and dynamic monitoring of the slope construction site. Figure 3 All cameras have output specifications of 1080p / 30fps or above, wide-angle camera 1 is located opposite to slope 4, telephoto camera 2 is located on both sides of slope 4 for supplementary shooting, and drone 3 cruise group regularly cruises to supplement the overlooking slope data.

[0054] The camera parameters involved are as follows:

[0055] Wide-angle camera (C1): monitoring range radius r1 = 50m, viewing angle θ1 = 120°;

[0056] Telephoto camera (C2, C3): monitoring range radius r2 = r3 = 30m, viewing angle θ2 = θ3 = 20°;

[0057] Drone camera (C4): monitoring range radius r4 = 70m, viewing angle θ4 = 45°;

[0058] Coverage area calculation:

[0059] 1) Calculate the coverage area A(C) of each camera i ): A(C i)=1 / 2×r1 2 θ i

[0060] Wide-angle camera C1: A(C1)=1 / 2×50 2 ×2π / 3≈2617.99m 2

[0061] Telephoto cameras C2, C3: A(C2) = A(C3) = 1 / 2 × 30 2 ×π / 9≈157.08m 2

[0062] Drone camera C4: A(C4)=1 / 2×70 2 ×π / 4≈1923.07m 2 ;

[0063] 2) Calculation of total coverage area:

[0064] 3) Overlap area calculation: Set the redundancy coefficient α to 0.1, there is a certain overlap between the cameras, and the average area of ​​the overlapping area is 10% of the coverage area: O(C i ,C j )=0.1×A(C i )

[0065] The total area of ​​the overlapping region is:

[0066]

[0067] 4) Check the redundancy conditions:

[0068]

[0069] According to the on-site inspection, the construction site area is 390.71 ≥ 0.10 × 3000 = 300, which is 3000m 2 ;

[0070] The overlap area is higher than the redundancy requirement and meets the requirement.

[0071] Step S3: Use the function Cv2 in the OpenCV library to adjust the image according to the distortion coefficient and camera parameters.

[0072] Enhance image quality. When extracting construction features, choose to use the pre-trained CNN model VGG-16, and use OpenCV and PyTorch to load the MiDaS model for depth estimation to obtain the slope excavation depth. The slope after correction in this embodiment is shown as follows Figure 4 As shown in Figure 5 As shown, Figure 5 It is concluded that the excavated part of the slope is about 5m lower than the unexcavated part.

[0073] Step S4: Synchronize the slope excavation depth of 5m to the preliminary slope digital twin model, and update the preliminary slope digital twin model to analyze the deformation and plastic strain distribution of the slope, such as Figure 6a and Figure 6b As shown, Figure 6a is a schematic diagram of cumulative displacement, Figure 6b is the plastic strain distribution diagram.

[0074] Step S5: Perform strength reduction calculation on the slope digital twin model through the finite element analysis software ABAQUS, analyze the safety factor of the slope after construction, and extract the slope safety factor diagram from Figure 7 It was concluded that the safety factor of the slope was greater than 1.8, much larger than the 1.3-1.5 specified in the specification. This slope was relatively safe during the construction process.

[0075] The present invention also provides a computer system, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception.

[0076] The present invention proposes a method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception. When establishing the digital twin model of the slope, the slope construction process is monitored in combination with multiple perspectives to establish a more accurate digital twin model of the slope. The multi-eye visual perception array avoids the shortcomings of previous monocular visual perception, and accurately and real-time provides slope construction progress pictures, which provides great help to the digital twin model of the slope construction. The combination of the multi-eye visual perception array and the digital twin model of the slope can provide a new direction for the intelligent construction of the slope, avoid special situations that occur during the slope construction process, and ensure the normal progress of the slope construction.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing a digital twin model of a roadbed slope based on multi-camera array visual perception, comprising the following specific steps: Step S1: Integrate the design drawings and construction planning documents of the slope construction, including topographic and geological data, engineering layout, and predetermined construction plan; use modeling tools to create a preliminary digital twin model of the slope based on the design drawings and construction plan; Step S2: According to the design drawings, construction plan and specific conditions on site, different types and numbers of cameras are set up, and their installation locations are planned to fully cover all key areas to form a multi-eye visual perception array; Step S3: Obtain an image of the slope changes during the construction process through a multi-eye visual perception array, and perform correction, enhancement and noise reduction on the image; use a deep learning algorithm to automatically identify and extract key construction features from the processed image, and match the construction feature data with preset monitoring parameters for subsequent analysis; Step S4: Based on the feature extraction results, the digital twin model is updated and refined using three-dimensional reconstruction technology; Step S5: Use the updated digital twin model to conduct detailed structural analysis and dynamic simulation, evaluate the problems and risks that may arise in different construction stages, and formulate emergency response strategies and preventive measures in combination with risk identification to address potential safety issues predicted by the analysis; Step S6: Continue to use the multi-eye visual perception array to perform real-time monitoring of the construction site.

2. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 1 is characterized in that: In step S2, the multi-view visual perception array includes multiple types of cameras deployed at preset positions and angles to form a spatially complementary perception network; each camera in the array not only works independently, but also provides multi-angle and multi-spectral visual perception capabilities through data fusion and collaborative processing.

3. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 2 is characterized in that: The cameras include wide-angle cameras, telephoto cameras, drone cameras and ordinary focal length cameras; the wide-angle cameras are deployed at a certain distance from the construction site to cover a large area and monitor the overall construction dynamics; the telephoto cameras are deployed near key construction areas to focus on capturing details of key construction points, and their focal lengths are adjusted according to actual construction needs; the drone cameras provide a dynamic perspective from an aerial perspective; the ordinary focal length cameras are deployed in blank areas of the array to improve the visual perception effect through cross-coverage of the field of view.

4. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 2 or 3, characterized in that: The multi-eye visual perception array C={C1, C2, ..., C n }, arrange the cameras, the position of each camera (x i ,y i ,z i ) and field of view θ i The arrangement needs to meet specific geometric and field of view coverage constraints; For each camera C i , which covers area A(C i ) is expressed as: Where: r i is the distance from the camera to the monitoring area, θ i is the camera’s viewing angle; In order to achieve seamless and comprehensive coverage of the entire construction site, the sum of the coverage areas of all cameras should not be less than the total area A of the construction site. total ,Right now: According to this formula, by adjusting the position and viewing angle θ of the camera i To achieve comprehensive monitoring of the entire site; assuming the total area of ​​the site is A total , then the total coverage area of ​​the multi-view visual perception array is not less than A total .

5. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 4 is characterized in that: The fields of view of multiple cameras should overlap appropriately: Define two cameras C i and C j The field of view overlap area O(C i ,C j )for: O(C i ,C j )=A(C i )∩A(C j ), In order to ensure redundancy, the total area of ​​the overlapping region is required to meet certain conditions: Where α is the redundancy coefficient, which is usually 0.1≤α≤0.

2.

6. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 1 is characterized in that: The step S3, the image correction, is to perform geometric correction and color correction; the image is processed using image enhancement technology to clearly highlight key construction features in the image.

7. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 1 is characterized in that: In step S3, a deep learning algorithm is used to select a pre-trained CNN model VGG-16 for transfer learning. The last few layers of the network are replaced and then these layers are trained on a specific data set with construction characteristics to adapt the model to a specific task. The MiDaS model is loaded using OpenCV and PyTorch for depth estimation to obtain the slope excavation depth.

8. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 1, characterized in that: In step S4, the changes of the three-dimensional model are synchronized to the finite element analysis software in real time to achieve real-time updating of the digital twin model.

9. The method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception according to claim 1, characterized in that: The updated digital twin model is analyzed in finite element analysis software, and it is found that the slope instability caused by construction is reduced to a certain threshold. The construction method or sequence is adjusted to reduce the risk, or specific safety measures are proposed, such as strengthening temporary supports to ensure the normal progress of construction.

10. A computer system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method for constructing a digital twin model of a roadbed slope based on multi-eye array visual perception as described in any one of claims 1-9.

Images