A multi-point multi-view video fusion method and system based on general plan

Abstract

The present invention discloses a multi-point multi-view video fusion method and system based on a general plan, which involves the fields of digital construction supervision, etc. The method includes: full-coverage monitoring point planning and equipment layout based on the general plan of the project; monitoring area division based on the height stratification of the building space; multi-angle video fusion in the local monitoring area; extraction of key areas of the local fusion video; regional filling fusion based on the project plan as the base map; rapid increase or decrease of camera images to fill in blind spots or remove duplicates; color balancing, seam removal and fusion optimization processing of multi-region video edges. The present invention effectively solves the dependence of multi-point multi-view video fusion on three-dimensional models and the cumbersomeness of frequent updates of three-dimensional models due to the complex and changeable construction site environment; realizes the synchronous supervision and dispatching command of multiple construction work surfaces, road surfaces, storage yards, etc. within the jurisdiction of the project, improves the supervision ability and efficiency; and has high practicality and universality.

Description

Technical Field

[0001] The present invention belongs to the technical fields of digital construction, smart construction, safe and green construction, video security supervision, etc., and specifically relates to image fusion processing, photogrammetry and virtual reality enhancement. More specifically, it relates to a multi-point and multi-view video fusion method and system based on a general plan. Background Art

[0002] Traditional construction site monitoring systems have always had problems such as fragmented video surveillance images, numerous and discontinuous systems, and an overall lack of spatial sense, and can no longer meet the supervision needs of modern construction production.

[0003] With the development of digital construction engineering, especially in the direction of digital intelligent construction, the application of big data and artificial intelligence in construction sites is becoming more and more extensive. At present, the more popular multi-point and multi-view surveillance video fusion on the market must rely on static three-dimensional models, such as oblique photography models, BIM models, etc. for projection mapping, and the surveillance cameras are all deployed at static points. Since the construction site is changing every day and every hour, if video fusion based on three-dimensional models is used, dynamic modeling of the building surface elevation, yard material stacking changes, etc. is required; the cost of high-frequency and multiple three-dimensional modeling is high; after the tower crane for setting up the camera monitoring group is lifted, the monitoring point parameters need to be manually adjusted; human intervention and maintenance are required in the later stage; real-time performance cannot be guaranteed; therefore, this technology cannot meet the increasingly dynamic construction site. However, if a three-dimensional model is not used, multi-point and multi-view, such as multiple tower cranes and different heights, cannot be spliced ​​and fused.

[0004] Therefore, how to effectively realize the overall plan video monitoring of intelligent digital construction sites is an urgent problem to be solved. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention proposes a multi-point and multi-view video fusion method and system based on a master plan, which can effectively realize the master plan video monitoring of an intelligent digital construction site.

[0006] To achieve the above object, according to one aspect of the present invention, a multi-point multi-view video fusion method based on a general plan is provided, comprising:

[0007] Analyze and plan the layout of monitoring equipment based on the general plan of the project, so that the monitoring equipment can fully cover the entire project site area;

[0008] Conduct spatial height work surface analysis on the general plan of the project and divide target areas with different spatial heights into different areas;

[0009] Retrieves the relevant monitoring images of each area for image feature point analysis, calculates the corresponding positions and transformation relationships between the monitoring devices corresponding to each area, preliminarily splices the multi-angle monitoring video images, and performs fusion processing on the spliced ​​images;

[0010] Each target area in the fused video surveillance image is cropped using image edge recognition;

[0011] Fill the cropped local fused video images into the corresponding area in the general plan of the project and fuse them;

[0012] Analyze the regionally filled and fused images, quickly add cameras to fill in the missing parts of the surveillance images, and quickly delete cameras to remove duplicates in the overlapping areas of the images;

[0013] The edges of multiple local fused video areas are subjected to overall color averaging and seam removal fusion optimization processing.

[0014] In some optional implementation schemes, the planning of the deployment points of monitoring equipment based on the analysis of the general plan of the engineering project includes:

[0015] Based on the analysis of the general plan of the project, the deployment points of the monitoring equipment are automatically planned so that the monitoring equipment can fully cover the entire project site area. After the monitoring is set up, the wired and / or wireless network deployment and adaptive networking are planned.

[0016] In some optional implementation schemes, the spatial height working surface analysis of the general plan of the engineering project and the regional division of target areas with different spatial heights include:

[0017] Conduct spatial height working surface analysis on the general plan of the project, divide target areas of different spatial heights on the construction site (such as construction surface, road surface, and storage yard) in the form of a lattice grid, and use target equipment (including tower cranes, construction elevators, lighting poles, etc.) to set up monitoring camera groups to divide the main construction surface and target areas where the construction space height will increase (such as construction surface, road surface, and storage yard).

[0018] In some optional implementation schemes, the relevant monitoring images of each area are retrieved for image feature point analysis, the corresponding positions and transformation relationships between the monitoring devices corresponding to each area are calculated, the multi-angle monitoring video images are preliminarily spliced ​​together, and the spliced ​​images are fused, including:

[0019] By calculating the corresponding position and image transformation relationship between monitoring devices at the same point but at different angles, the multi-angle monitoring video images are preliminarily spliced ​​together, and the spliced ​​images are fused.

[0020] In some optional implementation schemes, the process of cropping each target area in the fused video surveillance screen using image edge recognition includes:

[0021] The target area (such as construction surface, road surface, storage yard, etc.) in the partially fused video surveillance picture is processed by image edge recognition and cropping, and the cropped image is temporarily stored in the form of building block picture modules.

[0022] In some optional implementation schemes, the step of filling the cropped local fused video screen into a corresponding area in the general plan of the engineering project and fusing the image includes:

[0023] Integrate the partially fused and edge-cropped video into the corresponding block of the latitude and longitude grid of the general plan of the project. Parameters need to be set when using it for the first time. If there is no disassembly or change in the future, the monitoring can be unmanned.

[0024] In some optional implementation schemes, the analysis of the regionally filled and fused images, the rapid addition of cameras to fill in the missing surveillance images of the region, and the rapid deletion of cameras to remove duplicates in the overlapping areas of the images include:

[0025] For areas where surveillance footage is missing, cameras are quickly added to fill in the blind spots. For areas where footage overlaps, cameras are quickly removed to deduplicate. Building block-style assembly is used to fill in the gaps with movement and achieve rapid modular integration.

[0026] In some optional implementation schemes, the overall color averaging and seam removal fusion optimization processing of the edges of multiple local fused video regions includes:

[0027] Convert local fusion images from different angles into digital orthophoto DOM;

[0028] Use the blocks corresponding to the latitude and longitude grid of the general plan of the project and the scale information of the general plan, and the highest point monitoring of the tower crane as the reference terrain image;

[0029] The ASIFT algorithm is used to extract local fusion images from different angles as key frames and match points with the overall plan of the terrain image fused to the tower crane at the highest point.

[0030] The ASIFT algorithm is used to obtain the matching points between the key frame and the terrain image. ASIFT achieves complete affine invariance by calculating the angular changes of longitude and latitude angles. The matching points are the pixels with the same characteristics in the key frame image of the local fusion monitoring screen and the corresponding reference terrain image in the general plan.

[0031] According to another aspect of the present invention, a multi-point multi-view video fusion system based on a general plan is provided, comprising:

[0032] The full coverage point planning and equipment layout module is used to analyze and plan the layout points of monitoring equipment based on the general plan of the project, so that the monitoring equipment can fully cover the entire project site area;

[0033] The monitoring area division module is used to analyze the spatial height working surface of the general plan of the project and divide the target areas with different spatial heights;

[0034] The multi-angle video fusion module is used to retrieve the relevant monitoring images of each area for image feature point analysis, calculate the corresponding positions and transformation relationships between the monitoring devices corresponding to each area, preliminarily splice the multi-angle monitoring video images, and perform fusion processing on the spliced ​​images;

[0035] The local fusion video key area extraction module is used to perform image edge recognition and cropping processing on each target area in the fused video surveillance screen;

[0036] The regional filling and fusion module is used to fill the cropped local fusion video images into the corresponding areas in the general plan of the project and fuse them;

[0037] The module for quickly adding or removing cameras to fill in blind spots or remove duplicates is used to analyze the regional filled and fused images, quickly add cameras to fill in blind spots for the missing surveillance images in the area, and quickly remove cameras to remove duplicates for the overlapping areas of the images;

[0038] The multi-region video fusion optimization processing module is used to perform overall color averaging and seam removal fusion optimization processing on the edges of multiple local fusion video regions.

[0039] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0040] Taking the general plan of the project as the carrier, combined with video fusion, regional division, key area extraction of monitoring screens and building block screen splicing technology, dynamic and real-time acquisition of the general plan of the project at the construction site is realized. It effectively solves the dependence of multi-point and multi-view video fusion on three-dimensional models and the tediousness of frequent updates of three-dimensional models due to the complex and changeable construction site environment; the high cost of high-frequency and multiple three-dimensional modeling; the need to manually adjust the monitoring point parameters after the tower crane for setting up the camera monitoring group is lifted; human intervention and maintenance are required in the later stage; and real-time performance cannot be guaranteed. Based on the multi-point and multi-view video fusion of the general plan, low-cost, rapid deployment and unmanned duty in the later stage are achieved, with higher reliability and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of a method flow provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of a system architecture provided by an embodiment of the present invention;

[0043] Figure 3 It is a general plan of a project provided by an embodiment of the present invention;

[0044] Figure 4 This is a flow chart of a solution for quickly adding or reducing cameras and multi-layered fusion of videos for a project provided by an embodiment of the present invention;

[0045] Figure 5 It is a multi-angle video fusion in a local monitoring area of ​​a project provided by an embodiment of the present invention;

[0046] Figure 6 It is a project local fusion video key area extraction provided by an embodiment of the present invention;

[0047] Figure 7 It is a regional filling of a local fusion video screen of a project provided by an embodiment of the present invention;

[0048] Figure 8 The invention provides a multi-region video edge color uniformity and overall fusion optimization processing project. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0050] The present invention adopts advanced real-time video fusion technology, and fills and fuses multiple partially fused monitoring work surfaces, such as construction surfaces, road surfaces, and storage yards, based on the regional division and modular form of the master plan. It realizes the visualization of the construction site plane, which can better overlook the construction site; it realizes the dynamic real-time acquisition of the project master plan monitoring of the construction site, and through the monitoring and supervision of the project master plan, it realizes the simultaneous monitoring and dispatching command of multiple construction work surfaces, road surfaces, storage yards, etc. within the jurisdiction of the project, eliminating the need for users to switch monitoring screens for inspection, and improving monitoring capabilities and work efficiency.

[0051] The present invention relates to technical fields such as image fusion processing, photogrammetry, virtual reality enhancement, digital construction, smart construction, safe and green construction, and video security supervision, and discloses a multi-point and multi-view video fusion method and system based on a general plan. The method specifically comprises the following steps:

[0052] (1) Planning of monitoring points and equipment layout based on the overall plan of the project; (2) Division of monitoring areas based on the height of the building space; (3) Multi-angle video fusion within the local monitoring area; (4) Extraction of key areas of the local fused video; (5) Regional filling fusion based on the project plan as the base map; (6) Rapid increase or decrease of camera images to fill in blind spots or remove duplication; (7) Color averaging and seam removal fusion optimization processing of multi-region video edges.

[0053] The present invention realizes the dynamic real-time acquisition of the project master plan monitoring of the construction site, effectively solving the problems such as the dependence of multi-point multi-view video fusion on the three-dimensional model and the cumbersomeness of the frequent updating of the three-dimensional model due to the complex and changeable construction site environment; the high cost of high-frequency and multiple three-dimensional modeling; the need to manually adjust the monitoring point parameters after the tower crane for setting up the camera monitoring group is lifted; the need for human intervention and maintenance in the later stage; and the inability to ensure real-time performance. The multi-point multi-view video fusion method and system based on the master plan realizes low cost, rapid deployment and unmanned duty in the later stage, and has higher reliability and practicality.

[0054] Figure 1 1 is a flow chart of a multi-point multi-view video fusion method based on a general plan provided by an embodiment of the present invention, including:

[0055] 1. Full coverage monitoring point planning and equipment layout based on the general plan of the project;

[0056] 2. Division of monitoring areas based on the height of building space;

[0057] 3. Multi-angle video fusion within the local monitoring area;

[0058] 4. Extraction of key areas of local fusion video;

[0059] 5. Regional filling and fusion based on the project plan as the base map;

[0060] 6. Quickly add or subtract cameras to fill in blind spots or remove duplicates;

[0061] 7. Optimize the fusion processing of multi-area video edges by averaging colors and removing seams.

[0062] Figure 2 1 is an architecture diagram of a multi-point multi-view video fusion system based on a general plan provided by an embodiment of the present invention, including:

[0063] The full coverage point planning and equipment layout module is used to analyze and plan the layout points of monitoring equipment based on the general plan of the project, so that the monitoring equipment can fully cover the entire project site area;

[0064] The monitoring area division module is used to analyze the spatial height working surface of the general plan of the project and divide the construction surfaces, road surfaces, storage yards, etc. with different spatial heights into different areas;

[0065] The multi-angle video fusion module is used to retrieve the relevant monitoring images of each area for image feature point analysis, calculate the corresponding positions and transformation relationships between the monitoring devices corresponding to each area, preliminarily splice the multi-angle monitoring video images, and perform fusion processing on the spliced ​​images;

[0066] The local fusion video key area extraction module is used to use image edge recognition and cropping to process the construction surface, road surface, storage yard, etc. in the fused video surveillance screen;

[0067] The regional filling and fusion module is used to fill the cropped local fusion video images into the corresponding areas in the general plan of the project and fuse them;

[0068] The module for quickly adding or removing cameras to fill in blind spots or de-duplication is used to analyze the regionally filled and fused images, quickly add cameras to fill in blind spots for the missing surveillance images in the area, and quickly remove cameras to de-duplication for the areas where most of the images overlap;

[0069] The multi-region video fusion optimization processing module is used to perform overall color averaging and seam removal fusion optimization processing on the color difference and seams at the edges of multiple local fusion video regions.

[0070] Figure 3 It is the general plan of the project in the specific embodiment of the present invention; through the general plan of the project, the height of the building or building complex, the height and placement of the tower crane, the road planning, the yard layout and other important information are identified; the full coverage point planning and equipment layout module analyzes the important information identified, plans the layout points of the monitoring equipment, and after the monitoring is set up, plans the wired and wireless (Wi-Fi, 4G, 5G, etc.) network layout and adaptive networking. The layout plan of this embodiment: refer to Figure 5 The left part uses the tower crane as the installation fulcrum, with four sides in total. Two gun-type cameras are installed on each side in a horizontal coaxial manner. There are 8 2.8mm-12mm focal length 1080P infrared gun-type cameras; as the highest point monitoring, it mainly monitors the high-rise construction work surface of the building. A 15-meter-high pole is installed in the center of each of the four walls. Four gun-type cameras are installed on top of each pole, two of which are coaxial in a group and each group is installed back to back, monitoring roads and yards, etc. Through the layout of the monitoring equipment in the above embodiment, the entire project site area is initially fully covered.

[0071] Figure 4It is a flowchart of the project rapid increase or decrease of cameras and multi-layered video fusion solution in a specific embodiment of the present invention, involving two functional modules: a monitoring area division module and a rapid increase or decrease of camera image blindness or deduplication module; based on important information such as the height of the building or building complex, the placement of the tower crane, road planning, and yard layout, the monitoring area division module is used to analyze the spatial height working surface, and to divide the construction surfaces, road surfaces, and yards of different spatial heights. The rapid increase or decrease of camera image blindness or deduplication module analyzes the regionally filled and fused images, rapidly adds cameras to fill in the blind spots of the missing monitoring images in the area, and rapidly deletes cameras to deduplication in the areas where most of the images overlap.

[0072] Figure 5 It is the multi-angle video fusion in the local monitoring area of ​​the project in the specific embodiment of the present invention; Figure 5 The left part is the single multi-view images of the 8 gun cameras installed on the tower crane in the embodiment, and the right part is the local splicing image. In this process, the multi-angle video fusion module is used to retrieve the relevant monitoring images after regional division for image feature point analysis, calculate the corresponding positions and transformation relationships between the monitoring devices, preliminarily splice the multi-angle monitoring video images, and perform fusion processing on the spliced ​​images.

[0073] The main technologies used in the multi-angle video fusion module are: multi-channel image splicing is used to merge multiple related surveillance images; image edge fusion is used to process the splicing seams produced by image splicing; image color uniformity technology is used to unify the colors of multiple groups of surveillance images and solve the color difference caused by different exposure levels of multiple groups of surveillance; image affine and perspective transformation technology is used to transform the fused images of multiple groups of surveillance at multiple angles; image automatic correction technology is used to solve the deformation produced during the image transformation process. For example, there will be color difference, gaps and other differences at the image splicing. The optimization of the fusion algorithm can effectively solve the problems of local edge gaps; in global image fusion, due to the differences in shooting time, equipment shooting parameters, etc., the final fusion will have color differences in blocks. The global color can be unified through color extraction, correction, enhancement, etc.; image shooting, fusion, change of viewing angle, etc., will cause distortion. Contour detection, straight line detection, etc. can be adopted, and the structure of the curve and straight line structure constraint items is kept drawn, similarity transformation is constrained, the shape of the spliced ​​image is corrected, and the final image distortion caused by projection distortion is reduced.

[0074] Figure 6 It is the key area extraction of local fusion video in the specific embodiment of the project of the present invention; in this process, the key area extraction functional module of local fusion video is used, and the construction work surface is processed by image edge recognition and cropping through manual frame selection or deep learning; the cropped image picture is temporarily stored in the form of building block picture module.

[0075] Figure 7 It is the regional filling of the local fusion video screen of the project in the specific embodiment of the present invention; the regional filling fusion function module is used in this process. This module uses the monitoring screens of the construction work surface, road, yard, etc. that have been identified and cropped in the local fusion video screen to fill the corresponding areas in the general plan of the project, including the following steps: integrating the "multi-layered" "building block screen" (video with local fusion and edge cropping) into the block corresponding to the "latitude and longitude grid" of the general plan of the project; the general plan of the project has scale information, which effectively solves the problems of multi-point and multi-viewing angles (multiple tower cranes, different heights) that cannot be spliced ​​and fused without relying on a three-dimensional model, excessive distortion after multi-channel fusion, and "near big and far small" in imaging; parameters need to be set for the first use, and if there is no disassembly change in the later stage, the monitoring can be "unattended".

[0076] Figure 8 It is a project of multi-region video edge color uniformity and overall fusion optimization processing in a specific embodiment of the present invention; in this process, a multi-region video fusion optimization processing functional module is used to perform overall color uniformity, remove splicing seams, etc. on the edges of multiple local fusion video areas for optimization processing.

[0077] Multi-region video fusion optimization involves uniformly converting local fusion images of different angles into digital orthophoto maps (DOM). The blocks corresponding to the "latitude and longitude grid" of the general plan of the project and the scale information of the general plan are used, and the highest point monitoring of the tower crane (90 meters above the ground) is used as the reference terrain image; the ASIFT algorithm is used to extract the local fusion image videos of different angles as key frames and the general plan matching points of the fusion tower crane highest point monitoring image terrain image. The ASIFT algorithm is used to obtain the matching points of the key frame and the terrain image. ASIFT achieves complete affine invariance by calculating the angular changes of the longitude and latitude angles. The matching points are the pixel points with the same characteristics in the key frame image of the local fusion monitoring image and the relative reference terrain image in the general plan.

[0078] The coordinate transformation formula between the local fusion monitoring key frame image and the corresponding reference terrain image in the general plan is composed of the affine transformation matrix:

[0079]

[0080] Any matrix A can be decomposed into:

[0081]

[0082] Ri and Ti represent the transformation matrix corresponding to the rotation change and the transformation matrix corresponding to the tilt change, respectively.

[0083]

[0084] The local fusion monitoring key frame image after affine transformation is projected to the corresponding area of ​​the general plan through perspective transformation.

[0085] The general perspective transformation formula is

[0086] The multi-point multi-view video fusion method and system based on the master plan provided in the above embodiment, with the master plan of the engineering project as the carrier, combined with video fusion, area division, key area extraction of the monitoring screen, and "building block" screen splicing technology, realizes dynamic and real-time acquisition of the master plan monitoring of the engineering project at the construction site. It effectively solves the dependence of multi-point multi-view video fusion on the three-dimensional model and the cumbersomeness of frequent updates of the three-dimensional model due to the complex and changeable construction site environment; the high cost of high-frequency and multiple three-dimensional modeling; the need to manually adjust the monitoring point parameters after the tower crane for setting up the camera monitoring group is lifted; human intervention and maintenance are required in the later stage; and real-time performance cannot be guaranteed. The multi-point multi-view video fusion method and system based on the master plan realizes low cost, rapid deployment and unmanned duty in the later stage, and has higher reliability and practicality.

[0087] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0088] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-point multi-view video fusion method based on a general plan, characterized in that: include: Analyze and plan the layout of monitoring equipment based on the general plan of the project, so that the monitoring equipment can fully cover the entire project site area; Conduct spatial height work surface analysis on the general plan of the project and divide target areas with different spatial heights into different areas; Retrieves the relevant monitoring images of each area for image feature point analysis, calculates the corresponding positions and transformation relationships between the monitoring devices corresponding to each area, preliminarily splices the multi-angle monitoring video images, and performs fusion processing on the spliced ​​images; Each target area in the fused video surveillance image is cropped using image edge recognition; Fill the cropped local fused video images into the corresponding area in the general plan of the project and fuse them; Analyze the regionally filled and fused images, quickly add cameras to fill in the missing parts of the surveillance images, and quickly delete cameras to remove duplicates in the overlapping areas of the images; Perform overall color averaging and seam removal fusion optimization processing on the edges of multiple local fusion video areas; The above-mentioned spatial height working surface analysis of the general plan of the project and the regional division of target areas with different spatial heights include: Conduct spatial height work surface analysis on the general plan of the project, divide the target areas of different spatial heights on the construction site in the form of a latitudinal and longitude grid, and divide the target areas where the main construction surface and construction space height will increase by setting up target equipment for monitoring camera groups.

2. The method according to claim 1, characterized in that The planning of the layout of monitoring equipment based on the general plan of the engineering project includes: Based on the analysis of the general plan of the project, the deployment points of the monitoring equipment are automatically planned so that the monitoring equipment can fully cover the entire project site area. After the monitoring is set up, the wired and / or wireless network deployment and adaptive networking are planned.

3. The method according to claim 2, characterized in that The relevant monitoring images of each area are retrieved to analyze the image feature points, the corresponding positions and transformation relationships between the monitoring devices corresponding to each area are calculated, the multi-angle monitoring video images are preliminarily spliced ​​together, and the spliced ​​images are fused, including: By calculating the corresponding position and image transformation relationship between monitoring devices at the same point but at different angles, the multi-angle monitoring video images are preliminarily spliced ​​together, and the spliced ​​images are fused.

4. The method according to claim 3, characterized in that The method of using image edge recognition and cropping to process each target area in the fused video surveillance image includes: The target area in the locally fused video surveillance picture is cropped using image edge recognition, and the cropped image is temporarily stored in the form of a building block picture module.

5. The method according to claim 4, characterized in that Filling the cropped local fused video screen into the corresponding area in the general plan of the engineering project and fusing it includes: Integrate the partially fused and edge-cropped videos into the blocks corresponding to the latitude and longitude grid of the general plan of the project. Parameters need to be set when using it for the first time, and monitoring can be done unmanned in the future without disassembly and changes.

6. The method according to claim 5, characterized in that The analysis of the regionally filled and fused images, the rapid addition of cameras to fill in the missing surveillance images in the area, and the rapid deletion of cameras to remove duplicates in the overlapping areas of the images include: For areas where surveillance footage is missing, cameras are quickly added to fill in the blind spots, and for areas where footage overlaps, cameras are quickly removed to remove duplications.

7. The method according to claim 6, characterized in that The overall color averaging and seam removal fusion optimization processing of the edges of multiple local fusion video regions includes: Convert local fusion images from different angles into digital orthophoto DOM; Use the blocks corresponding to the latitude and longitude grid of the general plan of the project and the scale information of the general plan, and the highest point monitoring of the tower crane as the reference terrain image; The ASIFT algorithm is used to extract local fusion images from different angles as key frames and match points with the overall plan of the terrain image fused to the tower crane at the highest point. The ASIFT algorithm is used to obtain the matching points between the key frame and the terrain image. ASIFT achieves complete affine invariance by calculating the angular changes of longitude and latitude angles. The matching points are the pixels with the same characteristics in the key frame image of the local fusion monitoring screen and the corresponding reference terrain image in the general plan.

8. A multi-point multi-view video fusion system based on a general plan, characterized in that: include: The full coverage point planning and equipment layout module is used to analyze and plan the layout points of monitoring equipment based on the general plan of the project, so that the monitoring equipment can fully cover the entire project site area; The monitoring area division module is used to analyze the spatial height working surface of the general plan of the project, divide the target areas with different heights in the construction site space in the form of a longitude and latitude grid, and divide the target areas where the main construction surface and the construction space height will increase by setting up target equipment for monitoring camera groups; The multi-angle video fusion module is used to retrieve the relevant monitoring images of each area for image feature point analysis, calculate the corresponding positions and transformation relationships between the monitoring devices corresponding to each area, preliminarily splice the multi-angle monitoring video images, and perform fusion processing on the spliced ​​images; The local fusion video key area extraction module is used to perform image edge recognition and cropping processing on each target area in the fused video surveillance screen; The regional filling and fusion module is used to fill the cropped local fusion video images into the corresponding areas in the general plan of the project and fuse them; The module for quickly adding or removing cameras to fill in blind spots or remove duplicates is used to analyze the regional filled and fused images, quickly add cameras to fill in blind spots for the missing surveillance images in the area, and quickly remove cameras to remove duplicates for the overlapping areas of the images; The multi-region video fusion optimization processing module is used to perform overall color averaging and seam removal fusion optimization processing on the edges of multiple local fusion video regions.

Images