Real-time panoramic three-dimensional digital construction site map supervision system and method

Abstract

The present invention discloses a real-time panoramic three-dimensional digital construction site map supervision system and method. The system includes a data layer system, a core layer system, and an interaction layer system. The present invention realizes the fusion of multi-channel and multi-angle monitoring images; extracts key frame images, RTK, GPS, electronic compass and other information of the monitoring group in the form of time sequence, and combines the oblique photography real scene three-dimensional restoration technology to obtain a dynamically updated digital elevation model and orthophoto map of the construction site; fuses the orthophoto map with the satellite map, and realizes the rapid dispatching of the monitoring of engineering sites in multiple places around the world by the supervision department through the map indexing method, so as to have the ability of simultaneous remote guidance and supervision management for multiple engineering sites; multi-terminal interaction realizes the simultaneous collaborative supervision management of the same engineering site by multiple management departments, thereby improving the coherence and efficiency of cooperation among multiple departments.

Description

Technical Field

[0001] The present invention belongs to the field of construction site supervision, and particularly relates to a real-time panoramic three-dimensional digital construction site map supervision system and method. Background Art

[0002] In traditional digital construction sites, there are often problems such as scattered video surveillance images, numerous and discontinuous systems, lack of overall sense of space, building models being divorced from reality, inability to navigate inside the construction site, single system functions, and difficulty in event tracing, which can no longer meet the needs of modern construction production.

[0003] With the development of digitalization in construction projects, especially in the direction of digital intelligent construction, the application of big data and artificial intelligence in construction sites is becoming more and more extensive.

[0004] Currently, there is an urgent need for all-round video surveillance of intelligent digital construction sites to more effectively respond to emergencies at the construction site, monitor and store in a timely manner, and the data has the function of quickly positioning and tracing the whole process of events; high-precision real-scene three-dimensional models with geographical location information such as scale and longitude and latitude, and digital elevation models that can be used for earthwork measurement, visibility analysis, etc.; the function of quickly dispatching real-scene three-dimensional video surveillance images of engineering sites in different regions of the world to improve the ability of remote guidance and supervision and management of multiple engineering sites at the same time; and the function of multi-terminal collaborative interactive operation to improve the efficiency of real-time multi-party collaborative management at the construction site. Summary of the Invention

[0005] In view of the technical problems existing in the prior art, the present invention proposes a real-time panoramic three-dimensional digital construction site map supervision system and method, which adopts advanced real-time real-scene three-dimensional video fusion technology to highly restore the on-site construction environment and progress, and realizes cross-time and space, simultaneous supervision of multiple construction sites, and collaborative supervision of multiple departments in one construction site in a digital way; at the same time, the digital construction site map also provides managers with a "God's perspective" from multiple angles and across time and space of the construction site, realizing the visualization of the construction site plane, the traceability of the construction process, and the zero-distance of on-site command, and having a digital base map with other intelligent application bases such as digital elevation surveying and mapping, equipment remote control assistance, autonomous navigation, AR recognition and acceptance.

[0006] The present invention provides the following technical solutions:

[0007] A real-time panoramic three-dimensional digital construction site map supervision system is composed of a system architecture and a technical module architecture. Among them, the system architecture mainly includes: a data layer system, a core layer system, and an interaction layer system:

[0008] Data layer system (edge processing): Data is collected and encoded into groups through monitoring devices and then pushed to the video data stream processing platform; the data management platform manages the frame stream and stores the data; the cloud management platform extracts key frames to form a data stream and manages the network speed traffic.

[0009] Core layer system (data processing): The data obtained from the data layer system is stored in a database in the form of structured data, time-series data, and distributed data; the core algorithms mainly include: multi-angle multi-channel video fusion, real-scene three-dimensional reconstruction, digital elevation model, and digital orthophoto map.

[0010] Interaction layer system (terminal display): User management obtains the device usage management permission to quickly schedule the data of the core layer system; the scheduled data is used for seamless roaming viewing and multi-terminal collaborative interaction.

[0011] Among them, the technical module architecture mainly includes: data module, core algorithm module, and interaction supervision module:

[0012] S1. Part of the data module

[0013] S11. According to the design and planning drawings of the engineering project, automatically plan the layout points of monitoring and other devices to enable comprehensive monitoring coverage of the entire project construction area.

[0014] S12. After the monitoring is installed, perform adaptive networking with wired and wireless (Wi-Fi, 4G, 5G, etc.) networks.

[0015] S13. The front-end monitoring data needs to be time-synchronized.

[0016] S14. Extract the key frames of the monitoring video images.

[0017] S15. Manage and store the extracted key frames in a structured, time-series, and distributed database for dynamic data update.

[0018] S2. Part of the core algorithm module

[0019] S20. During the construction process, as the construction building grows taller, there will be monitoring blind spots or confidential areas, and it is necessary to increase or decrease the monitoring devices, and use adaptive layout control technology to network the added monitoring devices.

[0020] S21. Quickly fuse the added or removed camera images.

[0021] S22. Integrate the multi-angle multi-channel monitoring video images as a whole.

[0022] S23. Real-scene three-dimensional reconstruction.

[0023] S24. Digital elevation model.

[0024] S25. Orthophoto map for cross - time - space monitoring;

[0025] S26. Fusion of orthophoto map and satellite map.

[0026] S3. Interactive supervision module part

[0027] User management obtains device usage management permissions for fast scheduling of data for monitoring video fusion and satellite map fusion; the scheduled data is used for seamless roaming access and multi - terminal collaborative interaction; fast scheduling of monitoring of multi - location engineering project sites is achieved through satellite map indexing; multi - terminal interaction realizes simultaneous collaborative supervision and management of the same engineering site by multiple management departments.

[0028] Among them, the S2 core algorithm module also includes a core technical architecture, and the core technologies include: 1. Multi - path video fusion technology, 2. Video management add / delete fusion technology, 3. Automatic update technology of 3D base map, 4. Fast scheduling, seamless roaming, and multi - interaction technology.

[0029] Preferably, the multi - path video fusion technology in the core technology is a mature general - purpose technology module. This module is specifically used for: fusing multi - path monitoring pictures with relevance and splicing the fused multi - path images; performing image edge fusion on the splicing seams generated by image splicing; color - homogenizing the images of multi - group monitoring pictures to solve the color difference caused by different exposure degrees of multi - group monitoring; performing image affine and perspective transformation on the multi - group fused monitoring pictures for multi - angle transformation; automatically correcting the images deformed during the image transformation process.

[0030] Preferably, the video management add / delete fusion technology in the core technology is a self - developed technology module optimized directionally. This module is used for: the image self - calibration fusion technology analyzes the images of the newly added monitoring video pictures and the surrounding adjacent monitoring video pictures, and automatically calibrates and calculates the external parameter information such as the actual position of the newly added monitoring; the automatic fusion and automatic filling technology is used for: 1. Fusion of cameras added in key areas and uncovered points, 2. Filling the pictures of cameras deleted in non - important areas and confidential areas.

[0031] Preferably, the automatic update technology of 3D base map in the core technology is a self - developed technology module optimized directionally. This module is used for: in the reachable area of the monitoring pictures, extracting the key frames of the monitoring group and the monitoring GPS positions and other information at the latest same time node, and using oblique photography technology for real - scene 3D reconstruction; in the unreachable area of the monitoring pictures, such as the part blocked by buildings, using mobile devices such as drones and robotic dogs to regularly scan and fill the gaps in the real - scene 3D model; the real - scene 3D model has geographical location information such as scale and longitude and latitude, and the data accuracy is about 10 cm, which can be used for digital elevation models such as earthwork measurement and visibility analysis; the digital elevation model dynamically updated by time nodes can directly obtain the latest digital orthophoto map.

[0032] Preferably, the fast scheduling, seamless roaming, and multi-interaction technologies in the core technology are self-developed technology modules optimized directionally. This module is used for: integrating satellite map technology to fuse the dynamically updated digital orthophoto map with the satellite map, which is used to quickly schedule the real-time three-dimensional video monitoring images of engineering sites in different regions of the world, and realize the ability to remotely guide and supervise multiple engineering sites simultaneously with a brand-new function; the encoding group streaming technology is used to perform secondary encoding on the high-definition videos and digital elevation models of multiple monitors to improve the security of information transmission, and the secondary encoded data forms a data stream to improve the coherence and stability of the transmission of large amounts of information; the three-dimensional video analysis technology is used to analyze the large amounts of information in the encoded group stream and restore the secondary encoded information into high-definition video monitoring and digital elevation models, etc.; the multi-interaction operation technology adopts interaction methods such as gesture control, AR, and VR for multi-dimensional interaction and seamless roaming of three-dimensional videos; the multi-terminal collaboration technology is used to perform data synchronization management on video display terminals such as the command center large screen terminal, PC terminal, and mobile phone terminal, and determine the operation priority according to user permissions, so as to realize the collaborative operation and supervision management of the same engineering site data by multiple terminals.

[0033] The present invention also provides a real-time panoramic three-dimensional digital construction site map supervision device, which includes monitoring cameras, a storage array, a network, a video gateway, a decoding matrix, a central cluster server, mobile phones, PCs, and a command center large screen;

[0034] The monitoring cameras, storage array, and network are used to realize data collection, storage, and concurrent transmission of the S1 data module part; the video gateway is used to quickly schedule the data of the monitoring unit; the decoding matrix is used to parse and restore the secondary encoded information; the central cluster server is used to perform operations on the image algorithms of the S2 core algorithm module part; mobile phones, PCs, and the command center large screen are used for the display, interaction, and multi-party collaboration of the real-time panoramic three-dimensional monitoring scene.

[0035] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0036] With the full coverage of the communication network as the carrier (one network), combined with real-time three-dimensional, video fusion, and map association technologies (one map), the present invention builds a "real-time", "real-scene", "real-site" map platform (one screen), realizes the real-time three-dimensional video display that highly restores the on-site construction environment and progress, and realizes cross-time and space supervision of multiple construction sites simultaneously and collaborative supervision of multiple departments at one construction site in a digital way; at the same time, the digital construction site map provides a "God's perspective" with multiple angles and across time and space for managers; it realizes the visualization of the construction site plan, the traceability of the construction process, and the zero-distance on-site command, and has a digital base map with other intelligent application bases such as digital elevation surveying and mapping, remote control assistance for equipment, autonomous navigation, AR recognition and acceptance. Brief Description of the Drawings

[0037] Figure 1 is the system architecture diagram of the present invention;

[0038] Figure 2 is the technical architecture flowchart of the present invention;

[0039] Figure 3 is the technical principle architecture diagram of the present invention;

[0040] Figure 4 is the overall project layout design diagram in the specific embodiment of the present invention;

[0041] Figure 5 is the local tower crane layout installation diagram in the specific embodiment of the present invention;

[0042] Figure 6 is the digital orthophoto map of the project in the specific embodiment of the present invention. Detailed Description of the Preferred Embodiments

[0043] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0044] The present invention relates to technical fields such as image fusion processing, photogrammetry, video data and real - scene three - dimensional model fusion, virtual reality enhancement, data collaborative interaction, digital construction, intelligent construction, safe and green construction, and video security supervision. It discloses a real - time panoramic three - dimensional digital construction site map supervision system and method, and the method specifically includes the following steps: (1) Automatically plan the monitoring full - coverage points and equipment layout; (2) Adaptive networking technology; (3) Front - end data time synchronization; (4) Extract key frames of the monitoring video images; (5) Data dynamic update (structured, time - series, distributed database management); (6) Quickly add or subtract the fusion of camera images; (7) Multi - channel and multi - angle video fusion; (8) Dynamic update of real - scene three - dimensional reconstruction, digital elevation model, and orthophoto map (fused satellite map); (9) User management and equipment management; (10) Quick scheduling, seamless roaming and multi - terminal interaction.

[0045] The present invention realizes the fusion of multi-channel and multi-angle monitoring pictures; extracts key frame pictures, RTK, GPS, electronic compass and other information of the monitoring group in the form of time sequence, and combines the tilt photography real scene three-dimensional restoration technology to obtain the dynamically updated digital elevation model and orthophoto map of the construction site; fuses the orthophoto map with the satellite map, and realizes the rapid scheduling of the monitoring of engineering sites in many places around the world through the map indexing method for regulatory departments (such as the project command center of the bureau), so as to have the ability to remotely guide and supervise multiple engineering sites simultaneously; multi-terminal interaction realizes the collaborative supervision and management of the same engineering site by multiple management departments, thereby improving the coherence and efficiency of cooperation among multiple departments.

[0046] Figure 1 It is the system architecture diagram of a real-time panoramic three-dimensional digital construction site map supervision system provided by an embodiment of the present application. Refer to Figure 1 , the system includes: a data layer system, a core layer system, and an interaction layer system. Among them:

[0047] Data layer system (edge processing): Data is collected and encoded into a stream through monitoring devices and pushed to the video data stream processing platform; the data management platform performs frame stream management and data storage; the cloud management platform extracts key frames to form a data stream and manages network speed and traffic.

[0048] The hardware involved in the data layer system includes: video surveillance cameras, storage arrays, streaming media servers, and networks to implement Figure 2 the data collection, storage, and concurrent transmission of the S1 data module part in

[0049] Core layer system (data processing): The data obtained from the data layer system is stored in the database in the form of structured data, time-series data, and distributed data; the core algorithms mainly include: multi-angle and multi-channel video fusion, real scene three-dimensional reconstruction, digital elevation model, and digital orthophoto map.

[0050] The hardware involved in the core layer system includes: a video gateway for quickly scheduling monitoring unit data; a decoding matrix for parsing and restoring secondary encoded information; and a central cluster server for implementing Figure 2 the operation of the image algorithm in the S2 core algorithm module part in

[0051] Interaction layer system (terminal display): User management obtains the device usage management permission to quickly schedule the data of the core layer system; the scheduled data is used for seamless roaming viewing and multi-terminal collaborative interaction.

[0052] The hardware involved in the interaction layer system includes: gesture control devices, AR devices, VR devices, mobile phones, PCs, and the command center large screen, which are used to implement Figure 2The display, interaction, and multi-party collaboration of the real-time panoramic three-dimensional monitoring scenario in the S3 interaction supervision module.

[0053] Figure 2 This is a flowchart of a real-time panoramic three-dimensional digital construction site map supervision method provided by an embodiment of the present application. Refer to Figure 2 It mainly includes: a data module, a core algorithm module, and an interaction supervision module. Among them:

[0054] S1. Data module:

[0055] S11. According to the design and planning drawings of the engineering project, automatically plan the layout points of monitoring and other equipment to enable the monitoring to fully cover the entire project construction area; S12. After the monitoring is set up, perform adaptive networking with wired and wireless (Wi-Fi, 4G, 5G, etc.) networks; S13. The front-end monitoring data needs to be time-synchronized; S14. Extract the key frames of the monitoring video images; S15. Manage and store the extracted key frames in a structured, time-sequential, and distributed database for dynamic data updates.

[0056] S2. Core algorithm module:

[0057] S20. During the construction process, as the construction building increases in height, monitoring blind spots or confidential areas may appear. It is necessary to add or remove monitoring equipment, and use adaptive layout control technology to network the added monitoring equipment;

[0058] S21. Quick addition and removal of camera image fusion: Add monitoring equipment in key areas where monitoring needs to be enhanced and in coverage blind spots that appear after the construction building increases in height. The newly added monitoring images can be automatically fused into the global view; In non-important areas, confidentiality areas, etc., remove monitoring, and automatically fill in the missing parts of the base map for the generated missing parts;

[0059] S22. Overall fusion of multi-channel and multi-angle monitoring video images. Through image feature point analysis, calculate the corresponding positions and transformation relationships between the monitors, and initially splice multiple groups of monitoring images; Color differences and gaps may appear at the image splicing points. Through fusion algorithm optimization, local edge gap problems can be effectively solved; In global image fusion, due to different shooting times, device shooting parameters, etc., the final fusion will show color differences in units of blocks; Through color extraction, correction, enhancement, etc., the global color can be made consistent; Image shooting, fusion, perspective transformation, etc., will cause distortion; Take contour detection, line detection, etc., and maintain the structure of the curve and line structures with constraint terms, constrain the similarity transformation, correct the shape of the spliced image, and reduce the final image distortion caused by projection distortion;

[0060] S23. Monitor the reachable areas and perform real - scene 3D reconstruction to update the base map: mainly target the tower crane monitoring coverage area and the surrounding monitoring areas. Among them, the part covered by the tower crane is the main construction area. Extract the key frames of multiple - path monitoring, the key frames stored in step S15, GPS, electronic compass and other information, and at the same time, combine the current mature oblique photography technology and real - scene 3D restoration technology to achieve real - scene 3D reconstruction; for the areas where the monitoring pictures are unreachable, such as the parts blocked by buildings, use mobile devices such as drones and robotic dogs to regularly scan to fill in the gaps in the real - scene 3D model.

[0061] S24. The real - scene 3D model with GPS longitude and latitude geographical location, actual spatial scale, electronic compass and other information is the digital elevation model that can be used for earthwork measurement and visibility analysis; combined with the dynamic update of the data in step S15, the dynamically updated digital elevation model of the construction site is obtained to achieve cross - time - space monitoring.

[0062] S25. Through orthographic projection of the digital elevation model, an orthophoto map can be obtained.

[0063] S26. Integrate the orthophoto map with GPS longitude and latitude geographical location with the satellite map to achieve rapid dispatching of the monitoring of engineering sites in many places around the world by the regulatory department through the map indexing method.

[0064] S3. Interactive supervision module:

[0065] User management obtains the device usage management permission for the rapid dispatching of data for monitoring video integration and satellite map integration; the dispatched data is used for seamless roaming viewing and multi - terminal collaborative interaction; through the satellite map indexing method, rapid dispatching of the monitoring of engineering project sites in multiple places is achieved; multi - terminal interaction enables multiple management departments to simultaneously supervise and manage the same engineering site.

[0066] Figure 3 This is the technical principle architecture diagram of a real - time panoramic 3D digital construction site map supervision method and system provided by the embodiment of the present application. Refer to Figure 3 , the technical principle architecture mainly includes: 1. Multi - path video fusion technology, 2. Video management add / delete fusion technology, 3. 3D base map automatic update technology, 4. Rapid dispatching, seamless roaming and multi - interaction technology. Among them:

[0067] The multi-channel video fusion technology is a mature general technology module. The main technologies used in this module include: multi-channel image stitching for fusing multiple relevant surveillance video images; image edge fusion for processing the stitching seams generated by image stitching; image color uniformity technology for unifying the colors of multiple groups of surveillance videos and solving color differences caused by different exposure levels of multiple groups of surveillance videos; image affine and perspective transformation technologies for performing multi-angle transformation on the fused images of multiple groups of surveillance videos; and image automatic correction technology for solving the deformation generated during the image transformation process. For example, color differences and gaps may appear at the image stitching area, and local edge gaps and other problems can be effectively solved through the optimization of the fusion algorithm; in global image fusion, due to different shooting times, equipment shooting parameters, etc., color differences in the form of blocks may appear in the final fusion. Through color extraction, correction, enhancement, etc., the global colors can be made consistent; during image shooting, fusion, and perspective transformation, distortion may occur. Contour detection, line detection, etc. can be adopted, and the structure of curves and straight lines can be maintained by keeping the constraint terms. The similarity transformation can be constrained to correct the shape of the stitched image and reduce the distortion of the final image caused by projection distortion;

[0068] Video management can add / delete the fusion technology, which is a self-developed technology module for directional optimization. The main technologies used in this module include: image self-calibration fusion technology for analyzing the images of newly added surveillance video images and the surrounding adjacent surveillance video images, and automatically calibrating and calculating the external parameter information such as the actual position of the newly added surveillance; automatic fusion and automatic filling technology for: 1. fusing cameras in key areas and uncovered points, and 2. filling in the missing images of cameras in non-important areas and confidential areas.

[0069] The 3D base map automatic update technology is a self-developed technology module for directional optimization. For the reachable area of the surveillance video, mainly for the coverage area of tower crane surveillance and the surrounding surveillance area, where the covered part of the tower crane is the main construction area, the key frames of the surveillance group and the surveillance GPS positions, etc. at the latest same time node are extracted, and the oblique photography technology and the real scene 3D restoration technology are used for real scene 3D reconstruction; for the unreachable area of the surveillance video, such as the part blocked by buildings, mobile devices such as drones and robotic dogs are used for regular scanning to fill in the missing parts of the real scene 3D model; the real scene 3D model has geographical location information such as scale and longitude and latitude, and the data accuracy is about 10 cm, which can be used for digital elevation models such as earthwork measurement and visibility analysis; the latest digital orthophoto map can be directly obtained through the digital elevation model updated according to time nodes.

[0070] The rapid scheduling, seamless roaming, and multi-interaction technologies are self-developed technology modules optimized for orientation. The main technologies used in this module are: the integrated satellite map technology, which is used to integrate the dynamically updated digital orthophoto map with the satellite map. This technology is used to quickly schedule the real-time 3D video surveillance images of engineering sites in different regions of the world, enabling the ability to remotely guide and supervise multiple engineering sites simultaneously with a brand-new approach; the encoding and stream grouping technology is used to perform secondary encoding on the high-definition videos of multiple surveillance cameras and digital elevation models, etc., to improve the security of information transmission. The secondary encoded data forms a data stream to enhance the coherence and stability of the transmission of large amounts of information; the 3D video parsing technology is used to parse the large amounts of information in the encoded data stream and restore the secondary encoded information into high-definition video surveillance and digital elevation models, etc.; the multi-interaction operation technology adopts interaction methods such as gesture control, AR, and VR for multi-dimensional interaction and seamless roaming of 3D videos; the multi-terminal collaboration technology is used to manage data synchronization on video display terminals such as the command center large screen terminal, PC terminal, mobile phone terminal, etc., and determines the operation priority based on user permissions, realizing collaborative operation and supervision management of the data of the same engineering site by multiple terminals.

[0071] Figure 4 This is the overall layout design drawing of the project. Refer to Figure 4 , the following installation plan is designed: A total of 26 cameras are designed for the whole site, including 24 1080P infrared bullet cameras with a focal length of 2.8mm - 12mm and 2 dome cameras; using the tower crane as the installation fulcrum, there are four sides in total, and two bullet cameras are installed on each side in a horizontal coaxial manner to form a panoramic coverage, and one dome camera is installed at each diagonal to achieve detailed tracking. One 15-meter-high vertical pole is installed in the middle of each of the four perimeter walls, and four bullet cameras are installed above each vertical pole. Among them, two cameras are in a group and are installed back to back coaxially.

[0072] Figure 5 This shows the local tower crane layout installation drawing of the project. Refer to Figure 5 , there are three ways to install the monitoring on the tower crane: 1 at the front end of the tower crane boom, 2 at the end of the tower crane boom, and 3 in the middle of the tower crane body. Among them: For 3 in the middle of the tower crane body, Figure 4 As described above, two bullet cameras are installed on each side of the tower crane in a horizontal coaxial manner to form a panoramic coverage.

[0073] Image stitching and fusion are performed between adjacent multi-cameras, and there are the following relationships in feature matching:

[0074] SSD (Sum of Squared Distance)

[0075] (SSD is used for the best fit)

[0076] SAD (Sum of Absolute Difference)

[0077] (SAD is used to evaluate the similarity of the image module)

[0078] NCC (Normalized Cross Correlation)

[0079] (NCC is used for normalization)

[0080] At the front end 1 and the end 2 of the boom of the tower crane, multi - vision panoramic spherical monitoring cameras equipped with RTK, GPS, and electronic compasses are installed. During the operation of the boom, cameras 1 and 2 will move in a circular motion along with the boom. The multi - vision cameras of the cameras simultaneously collect images from multiple different angles such as vertical and inclined, obtaining rich high - resolution textures of the top and side views of the building; combining the information of RTK, GPS, electronic compasses, etc. on monitoring cameras 1 and 2, using oblique photography technology, comprehensively perceive complex scenes in a large - scale, high - precision, and high - definition manner, truly reflect the ground object situation, accurately obtain object - side texture information, and generate real - life three - dimensional models. By fusing the information of RTK, GPS, electronic compasses, etc., a digital elevation model with attributes such as the appearance, position, and height of the ground objects intuitively reflected is generated. The digital elevation model is used to ensure the real effect and survey - level accuracy.

[0081] During the operation of the boom of the tower crane, the movement of the boom is at a non - uniform speed. The accuracy of the information such as RTK, GPS, and electronic compasses on monitoring cameras 1 and 2 affects the generation efficiency and accuracy of the model. In the improved algorithm, Kalman filtering's "prediction + correction" is used for optimization to improve the survey - level accuracy of the digital elevation model, which is used to meet the requirements of earthwork measurement, visibility analysis, etc.

[0082] Figure 6 For the project digital orthophoto map in the specific embodiment, the digital elevation model updated dynamically according to time nodes can directly obtain the latest digital orthophoto map. Fusing the dynamically updated digital orthophoto map with the satellite map is used to quickly dispatch the real - life three - dimensional video monitoring images of the engineering sites in different regions of the world, realizing the ability to remotely guide and supervise multiple engineering sites simultaneously with a brand - new approach.

[0083] Finally, combining gesture control, AR, VR and other interaction methods adopted by multi - interaction operation technology for three - dimensional video multi - dimensional interaction and seamless roaming; multi - terminal collaboration technology is used for data synchronization management on video display terminals such as the large - screen terminal of the command center, PC terminal, mobile phone terminal, etc., and the operation priority is determined by user permissions, realizing the collaborative operation and supervision management of the same engineering site data by multiple terminals.

[0084] A real-time panoramic three-dimensional digital construction site map supervision method and system provided in the above embodiments uses full coverage of the communication network as the carrier (one network), combines real scene three-dimensional, video fusion, and map association technologies (one map), and builds a "real-time", "real scene", and "real site" map platform (one screen) to solve problems such as fragmentation, picture segmentation, and narrow viewing angles in traditional video surveillance, and realizes a real-time real scene three-dimensional video display that highly restores the on-site construction environment and progress, and realizes cross-time and space supervision of multiple construction sites at the same time and collaborative supervision of multiple departments at one construction site in a digital manner; at the same time, the digital construction site map provides a "God's perspective" of multiple angles and cross-time and space for managers; realizes the visualization of the construction site plane, the traceability of the construction process, and zero-distance on-site command, and has a digital base map with other intelligent application bases such as digital elevation surveying and mapping, equipment remote control assistance, autonomous navigation, AR recognition and acceptance.

[0085] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A real-time panoramic three-dimensional digital construction site map supervision system, characterized in that, Including: A data layer system, which collects and encodes data streams through monitoring devices and pushes them to a video data stream processing platform. A data management platform connected to the video data stream processing platform manages frame streams and stores data. A cloud management platform connected to the data management platform extracts key frames to form data streams and manages network speed traffic. A core layer system, which stores the data obtained from the data layer system in a database in the form of structured data, time-series data, and distributed data, and performs image processing. Image processing includes multi-angle and multi-channel video fusion, real-scene 3D reconstruction, digital elevation model, and digital orthophoto map. An interaction layer system, which obtains device usage management permissions from user management and then performs rapid scheduling of the data of the core layer system. The scheduled data is used for seamless roaming viewing and multi-terminal collaborative interaction. A real-time panoramic 3D digital construction site map supervision method implemented using this system, including the following steps: S2. Data processing S20. During the construction process, as the construction building increases in height, monitoring blind spots or confidential areas may appear. It is necessary to add or remove monitoring devices, and use adaptive layout control technology to form a network for the added monitoring devices. S21. Fast addition and removal of camera image fusion: In key areas where monitoring needs to be enhanced and coverage blind spots that appear after the construction building increases in height, add monitoring devices, and the newly added monitoring images are automatically fused into the global view. In non-important areas and confidential areas, remove monitoring, and automatically fill in the missing parts of the base map for the resulting missing parts. S22. Overall fusion of multi-channel and multi-angle monitoring video images: Through image feature point analysis, calculate the corresponding positions and transformation relationships between monitoring devices, initially splice multiple groups of monitoring images, and perform fusion processing on the spliced images. S23. Perform real-scene 3D reconstruction to update the base map. S24. The real-scene 3D model with GPS longitude and latitude geographical location, actual spatial scale, and electronic compass information is the digital elevation model used for earthwork measurement and visibility analysis. Combine the data in step S15 to dynamically update the digital elevation model to achieve cross-time and space monitoring. S25. The digital elevation model obtains a digital orthophoto map through orthographic projection. S26. Fuse the orthophoto map with GPS longitude and latitude geographical location with the satellite map to achieve rapid scheduling of the monitoring of engineering sites in multiple locations around the world by regulatory authorities through the map indexing method.

2. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 1, wherein The data layer system includes video surveillance cameras, storage arrays, streaming media servers, and networks. Among them, the video surveillance cameras are used for data collection, the storage arrays are used for data storage, and the streaming media servers and networks are used for data concurrent transmission. The core layer system includes video gateways, decoding matrices, and central cluster servers. Among them, the video gateways are used for rapid scheduling of monitoring data, the decoding matrices are used for parsing and restoring encoded information, and the central cluster servers are used for image processing. The interaction layer system includes multi-terminals. The multi-terminals are used to realize the display, interaction, and multi-party collaboration of real-time panoramic 3D monitoring scenarios.

3. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 1 or 2, characterized in that, The multi-terminals include gesture control devices, AR devices, VR devices, mobile phones, PCs, and command center large screens.

4. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 1, wherein, The real-time panoramic three-dimensional digital construction site map supervision method implemented by using this system further includes the following steps: S1. Data collection and storage S11. Automatically plan the layout points of monitoring devices so that the monitoring devices can comprehensively cover the entire project construction area; S12. After the monitoring devices are erected, perform adaptive networking; S13. Perform time synchronization processing on the monitoring data; S14. Extract the key frames of the monitoring video images; S15. Manage and store the extracted key frames in a structured, sequential, and distributed database for data dynamic update; S3. Interactive supervision User management obtains the device usage management permission for rapid scheduling of data for monitoring video fusion and satellite map fusion; the scheduled data is used for seamless roaming viewing and multi-terminal collaborative interaction; rapid scheduling of monitoring of engineering project sites in multiple locations is achieved through the satellite map indexing method; multi-terminal interaction enables multiple management departments to simultaneously conduct collaborative supervision and management of the same engineering site.

5. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 4, characterized in that, Perform adaptive networking through a wired or wireless network. The wireless network includes Wi-Fi, 4G, or 5G.

6. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 4, characterized in that, The overall fusion of multi-channel and multi-angle monitoring video images includes: Multi-channel image stitching, which is used to fuse multiple relevant monitoring images; Image edge fusion, which is used to process the stitching seams generated by image stitching; Image color equalization technology, which is used to unify the colors of multiple groups of monitoring images and solve the color difference caused by different exposure degrees of multiple groups of monitoring; Image affine and perspective transformation technology, which is used to perform multi-angle transformation on the fused images of multiple groups of monitoring; Image automatic correction technology, which is used to solve the deformation generated during the image transformation process.

7. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 4, characterized in that, Rapid addition and deletion of camera image fusion includes: Image self-calibration fusion technology, which is used to perform image analysis on the newly added monitoring video image and the surrounding adjacent monitoring video images, and automatically calibrate and calculate the external parameter information of the newly added monitoring device; Automatic fusion and automatic filling technology, which is used for the fusion of cameras added in key areas and uncovered points, and the filling of the images of cameras deleted in non-important areas and confidential areas.

8. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 4, characterized in that Real scene three-dimensional reconstruction includes: In the reachable area of the monitoring image, extract the key frames of the monitoring group and the GPS position information of the monitoring devices at the latest same time node, and use oblique photography technology for real scene three-dimensional reconstruction; In the unreachable area of the monitoring image, use mobile devices to regularly scan and complete the missing parts of the real scene three-dimensional model; The real scene three-dimensional model with GPS latitude and longitude geographical location, actual spatial scale, and electronic compass information is the digital elevation model used for earthwork measurement and visibility analysis; the latest digital orthophoto map can be directly obtained through the digital elevation model dynamically updated by time node.

9. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 8, characterized in that, The unreachable area of the monitoring image includes the part blocked by buildings, and the mobile devices include drones and robotic dogs.

10. The real-time panoramic three-dimensional digital construction site map supervision system according to claim 4, characterized in that Rapid scheduling, seamless roaming, and multi-terminal interaction include: Fusion satellite map technology, which is used to fuse the dynamically updated digital orthophoto map with the satellite map. This technology is used to rapidly schedule the real scene three-dimensional video monitoring images of engineering sites in different regions of the world, and realize the ability to remotely guide and supervise multiple engineering sites simultaneously with a brand new perspective. Coding group flow technology is used to perform secondary coding on high-definition videos and digital elevation models monitored by multiple channels to improve the security of information transmission. The secondary coding forms a data stream to improve the coherence and stability of large-volume information transmission; Three-dimensional video analysis technology is used to analyze the large-volume information of the coding group flow and restore the secondary coding information into high-definition video monitoring and digital elevation models, etc.; Multi-interaction operation technology adopts interaction methods such as gesture control, AR, and VR and is used for multi-dimensional interaction and seamless roaming of three-dimensional videos; Multi-terminal collaboration technology is used to perform data synchronization management on the video display end and determine the operation priority according to user permissions to achieve collaborative operation and supervision management of the same project site data by multiple terminals; the video display end includes the command center large screen end, PC end, and mobile phone end.

Images