3D WebGIS video fusion method under weak constraint condition

Abstract

The weak constraint 3D WebGIS video fusion method belongs to the technical field of virtual video fusion, and is characterized in that: including the following steps: S1, three-dimensional model construction; S2, three-dimensional model optimization adjustment; S3, 3D WebGIS platform construction; S4, access to video stream data; S5, model view matrix and projection matrix calculation; S6, view cone construction and occlusion detection; S7, rendering based on WebGL fragment shader; S8, video image dynamic texture mapping; S9, video projection edge feathering; S10, dynamic parameter adjustment. The weak constraint 3D WebGIS video fusion method solves the problem of video projection occlusion to a certain extent, realizes video fusion under weak constraint, improves the video fusion effect, and has high practicability, applicability and adaptability.

Description

Technical Field

[0001] A 3D WebGIS video fusion method under weak constraints belongs to the field of virtual video fusion technology. Background Technology

[0002] The representation of traditional geospatial information is an important research area in geographic information science, representing the process of reproducing geospatial information. With the development of computer graphics, the combination of video and 3D (3D) GIS has become a new way of representing geographic information. Currently, although it is possible to map video to a 3D model video area using projection texture mapping principles, issues arise such as video images being occluded by 3D model structures. Furthermore, there are problems with the precise calculation of parameters such as camera position, orientation, and field of view required for combining video and 3D GIS, resulting in poor fusion effects between video information and the 3D spatial scene. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a 3D WebGIS video fusion method under weak constraints that solves the video projection occlusion problem, realizes video fusion under weak constraints, and improves the video fusion effect.

[0004] The technical solution adopted by this invention to solve its technical problem is: a 3D WebGIS video fusion method under weak constraints, characterized by the following steps:

[0005] S1, 3D model construction;

[0006] S2, 3D model optimization and adjustment;

[0007] S3, 3DWebGIS platform construction;

[0008] S4, access video stream data;

[0009] S5, Calculation of model-view matrix and projection matrix;

[0010] S6, View frustum structure and occlusion detection;

[0011] S7, based on WebGL fragment shader rendering;

[0012] S8, Dynamic Texture Mapping for Video Images;

[0013] S9, video projection edge feathering;

[0014] S10, dynamic parameter tuning.

[0015] Preferably, the construction of the three-dimensional model includes the following steps:

[0016] S1.1, using the method of UAV aerial oblique photogrammetry modeling, the flight path and altitude of the UAV are planned according to the video area to ensure that the flight range covers the video projection area and complete the aerial triangulation.

[0017] S1.2, Use a drone to fly along a prescribed route and collect multi-angle photos of the ground in the modeling area, with the photos in JPG format;

[0018] S1.3. Use the model generation tool Context Capture 2018 to create a reconstruction project, import the entire photo file data, submit aerial triangulation, process it using the Context Capture engine, select a spatial reference system that is consistent with the spatial reference of the aerial triangulation control points, select a regular planar mesh for construction and segmentation, and automatically map textures and output OBJ format model data for 3D modeling.

[0019] Preferably, the optimization and adjustment of the three-dimensional model includes the following steps:

[0020] S2.1 Optimize the surface of the constructed 3D model, and remove factors that affect the projection effect in the video area of ​​the 3D model without affecting the overall effect of the model.

[0021] Step 2.2: Use the Cesiumlab format conversion tool to convert the model file obj format into the 3DTiles format supported by the Cesium engine.

[0022] Preferably, the method further includes selecting the open-source Cecium digital earth engine to build a 3D WebGIS virtual geographic environment, loading satellite image data into Cesium as the virtual environment base map, loading 3DTiles format model data into Cesium to form a virtual three-dimensional geographic scene, and adjusting the spatial position of the three-dimensional model in the virtual environment so that the three-dimensional model coincides with the real geographical location in the satellite image.

[0023] Preferably, the method further includes supporting access to local video files and HTTP protocol network video data, and utilizing HTML5 in a Web environment. <video>The tag reads local video data.

[0024] Preferably, the method further includes calculating the Camera-type view matrix and the inverse view matrix using viewpoint coordinates, azimuth angle, pitch angle, and roll angle;

[0025] Reverse the result and multiply it by the inverse view matrix to obtain the inverse view projection matrix;

[0026] The view projection matrix is ​​obtained by inverting the view projection matrix.

[0027] Generate a planar standard model matrix using the projection distance and matrix;

[0028] Multiplying the standard planar model matrix with the inverse view matrix yields the model matrix required for the transformation.

[0029] Preferably, the frustum construction and occlusion detection include the following steps:

[0030] S6.1, using parameters such as field of view, focal length, and projection distance to create a view frustum, first determine the camera position and orientation, generate the view frustum geometry, then set the view frustum pitch and yaw angle parameters, and finally draw the view frustum;

[0031] S6.2 enables depth caching, performs occlusion detection on the 3D model, and crops the currently occluded and invisible model areas.

[0032] Preferably, the method further includes calculating texture coordinates in the GPU fragment shader through projected texture mapping, including the following steps:

[0033] S7.1, transform the vertices of the 3D model from the model coordinate system to the camera coordinate system through the model-view matrix;

[0034] S7.2, transform to world coordinate system using the camera view inverse matrix;

[0035] S 7.3, transform it to the projection coordinate system according to the projection view matrix;

[0036] S 7.4, transform to the clipping coordinate system using the projection matrix;

[0037] S 7.5, normalize the obtained projection vertex coordinates to the [0, 1] interval to obtain the projection texture coordinates.

[0038] Preferably, the method further includes fusing the static texture of the 3D model surface with the dynamic video texture, making the dynamic video texture the topmost texture, multiplying the camera view matrix and the inverse view projection matrix in the virtual scene to obtain a cube model view, and similarly obtaining an inverse cube model view, and assigning the two model views to the consistency mapping parameters of the classification primitive class to realize the projection of the video image on the 3D scene.

[0039] Preferably, the method further includes,

[0040] Access the video to the web environment by inputting the video data address;

[0041] The camera position in the virtual 3D environment can be controlled in three ways: by clicking the mouse to select the camera position and obtaining the viewpoint coordinates, by projecting according to the current viewpoint, and by inputting the camera's geographical location coordinates.

[0042] The video projection area is dynamically adjusted by adjusting parameters such as horizontal angle, vertical angle, yaw angle, pitch angle and projection distance to adjust the size, direction and distance of the projected video image, so that the video and the 3D model can be highly integrated.

[0043] Compared with the prior art, the beneficial effects of this invention are:

[0044] This 3D WebGIS video fusion method under weak constraints constructs a 3D model. Without affecting the overall model effect, it removes factors that may affect the projection effect, such as movable vehicles and trees, from the 3D model video area. It optimizes the surface of the 3D model in the video coverage area, providing a foundation for the fusion of video and 3D model. Then, it uses projection texture mapping to project the incoming video frames onto the video model area in the 3D geographic scene. Finally, it uses dynamic parameter adjustment on the Web end to optimize the video projection. This method achieves effective fusion of the monitored video stream data and the 3D model data of the monitored scene. The real-time nature of the video stream and the spatial nature of 3D GIS complement each other, achieving consistency between the video scene and the 3D model in time and space. It solves the video projection occlusion problem to a certain extent, realizes video fusion under weak constraints, and improves the video fusion effect. At the same time, this invention has high practicality, applicability, and adaptability. Attached Figure Description

[0045] Figure 1 A technical flowchart illustrating the method of this invention. Detailed Implementation

[0046] Figure 1 This is the preferred embodiment of the present invention, which is described below in conjunction with the accompanying drawings. Figure 1 The present invention will be further described below.

[0047] A 3D WebGIS video fusion method under weak constraints includes the following steps:

[0048] S1, 3D model construction.

[0049] The components of a 3D model include the following steps:

[0050] S1.1, using the method of drone aerial oblique photography modeling, the drone flight path and altitude are planned according to the video area to ensure that the flight range covers the video projection area;

[0051] S 1.2, Use a DJI Air2 drone to fly along a prescribed route and collect multi-angle photos (JPG format) of the ground in the modeling area;

[0052] S 1.3, use the model generation tool Context Capture 2018 to create a reconstruction project, import the entire photo file data, submit aerial triangulation, process it using the Context Capture engine, select a spatial reference system consistent with the spatial reference of the control points, select a regular planar mesh for construction and segmentation, and automatically map textures and output OBJ format model data for 3D modeling.

[0053] S2, 3D model optimization and adjustment.

[0054] The optimization and adjustment of the 3D model includes the following steps:

[0055] S2.1, use Dp Modeler V2.3 software to make detailed corrections to the 3D model, optimize the model surface of the constructed model, and remove factors that may affect the projection effect, such as movable vehicles and trees, in the video area of ​​the 3D model without affecting the overall effect of the model; in this implementation case, the area of ​​trees and vehicles removed from the model video area is 466.2 square meters.

[0056] S2.2, perform format conversion using the Cesiumlab conversion tool to convert the model file obj format to the 3DTiles format supported by Cesium. Specifically, convert the glTF and json file groups containing geometric and semantic information into a single 3D data file, b3dm; add a data description file tilesset.json, and convert b3dm into 3DTiles data.

[0057] S3, 3DWebGIS platform construction.

[0058] Specifically, the open-source Cecium digital earth engine was chosen to build a 3D WebGIS virtual geographic environment. Satellite frontal image data was loaded into Cesium as the base layer, and 3DTiles format model data was loaded into Cesium to form a virtual three-dimensional geographic scene. The viewpoint was fixed to the video projection area.

[0059] S4, access video stream data.

[0060] Specifically, the video used supports access to local video files and HTTP protocol network video data, utilizing HTML5 in the web environment. <video>The tag reads local video data, and the required video data is obtained by using the unmanned aerial vehicle to take pictures at a specific point.

[0061] S5, model view matrix and projection matrix calculation.

[0062] Specifically, the view matrix and the inverse view matrix of the Camera class are calculated by using the viewpoint coordinates, the azimuth angle, the pitch angle and the roll angle; the inverse view projection matrix is obtained by reversing the result and multiplying it by the inverse view matrix; the view projection matrix is obtained by reversing the inverse view projection matrix; the plane standard model matrix is generated by using the projection distance and the matrix; and the model matrix required for transformation is obtained by multiplying the plane standard model matrix by the inverse view matrix.

[0063] S6, frustum construction and occlusion detection.

[0064] The frustum construction and occlusion detection include the following steps:

[0065] S6.1, a frustum is created by using the field of view angle, the focal length, the projection distance and other parameters, the camera position and direction are first determined, the frustum geometry is generated, then the frustum pitch angle and yaw angle parameters are set, and finally the frustum is drawn, and the parameters in the embodiment are as follows: geographic coordinates (longitude: 117.9931246°, latitude: 36.8081395°, height: 56m), yaw angle (98.5°), pitch angle (-37.8°), roll angle (24.4°), projection distance (65m), field of view angle (34.1°);

[0066] S6.2, the depth buffer is enabled, the occlusion detection is performed on the three-dimensional model, and the model area that is currently occluded and invisible is cropped.

[0067] S7, WebGL fragment shader rendering.

[0068] Specifically, the WebGL fragment shader rendering is to calculate the fragment texture coordinates, the texture coordinates are calculated in the GPU fragment shader by projection texture mapping, and the following steps are included:

[0069] S7.1, the model vertex is converted from the model coordinate system to the camera coordinate system through the model view matrix;

[0070] S 7.2, it is converted to the world coordinate system through the camera view inverse matrix;

[0071] S 7.3, it is converted to the projection coordinate system according to the projection view matrix;

[0072] S 7.4, it is converted to the clipping coordinate system through the projection matrix;

[0073] S 7.5, the obtained projection vertex coordinates are normalized to the [0, 1] interval to obtain the projection texture coordinates.

[0074] S8, Dynamic Texture Mapping for Video Images.

[0075] Specifically, the static texture of the 3D model surface is fused with the dynamic video texture, and the dynamic video texture is made to cover the top layer texture to realize the positioning of the 3D model and the display of real-time scene video images. That is, the camera view matrix in the virtual scene is multiplied with the inverse view projection matrix to obtain the cube model view, and the inverse cube model view is obtained in the same way. The two model views are then assigned the consistency mapping parameters of the classification primitive class to realize the projection of video images onto the 3D scene.

[0076] S9, video projection edge feathering.

[0077] Specifically, enabling softShadows (Percentage-Closer-Filtering, or PCF for short, is commonly used to soften ShadowMap edges) and optimizing shaders (using 3D model texture coordinates for the video projection boundary area) weakens video projection edge artifacts.

[0078] S10, dynamic parameter tuning.

[0079] Input the video data address to connect the video to the Web environment; input the camera's geographical coordinates (longitude, latitude, and altitude); dynamically adjust parameters such as viewpoint position, horizontal angle, vertical angle, yaw angle, pitch angle, and projection distance to control the size, direction, and distance of the projected video image, so that the video and the 3D model can be highly integrated and optimized to match the video image with the spatial position of the virtual 3D geographic scene.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.< / video> < / video>

Claims

1. A 3D WebGIS video fusion method under weak constraints, characterized in that: Includes the following steps: S1, 3D model construction; S2, 3D model optimization and adjustment; S3, 3DWebGIS platform construction; S4, access video stream data; S5, Calculation of model-view matrix and projection matrix; S6, View frustum structure and occlusion detection; S7, based on WebGL fragment shader rendering; S8, Dynamic Texture Mapping for Video Images; S9, video projection edge feathering; S10, dynamic parameter tuning; The aforementioned cone structure and occlusion detection include the following steps: S6.1, using the field of view, focal length, and projection distance parameters to create the view frustum, first determine the camera position and orientation, generate the view frustum geometry, then set the view frustum pitch and yaw angle parameters, and finally draw the view frustum; S6.2 enables depth caching, performs occlusion detection on the 3D model, and crops the currently occluded and invisible model areas; The method also includes calculating the Camera-type view matrix and the inverse view matrix using viewpoint coordinates, azimuth angle, pitch angle, and roll angle; Reverse the result and multiply it by the inverse view matrix to obtain the inverse view projection matrix; The view projection matrix is ​​obtained by inverting the view projection matrix. Generate a planar standard model matrix using the projection distance and matrix; Multiply the standard planar model matrix by the inverse view matrix to obtain the model matrix required for the transformation; The method also includes, Access the video to the web environment by inputting the video data address; The camera position in the virtual 3D environment can be controlled in three ways: by clicking the mouse to select the camera position and obtaining the viewpoint coordinates, by projecting according to the current viewpoint, and by inputting the camera's geographical location coordinates. The video projection area is dynamically adjusted by adjusting the horizontal angle, vertical angle, yaw angle, pitch angle and projection distance parameters to adjust the size, direction and distance of the projected video image, so that the video and the 3D model can be highly integrated.

2. The 3D WebGIS video fusion method under weak constraints according to claim 1, characterized in that: The construction of the three-dimensional model includes the following steps: S1.1, using the method of UAV aerial oblique photogrammetry modeling, the flight path and altitude of the UAV are planned according to the video area to ensure that the flight range covers the video projection area and complete the aerial triangulation. S1.2, Use a drone to fly along a prescribed route and collect multi-angle photos of the ground in the modeling area, with the photos in JPG format; S1.

3. Use the model generation tool Context Capture 2018 to create a reconstruction project, import the entire photo file data, submit aerial triangulation, process it using the Context Capture engine, select a spatial reference system that is consistent with the spatial reference of the aerial triangulation control points, select a regular planar mesh for construction and segmentation, and automatically map textures and output OBJ format model data for 3D modeling.

3. The 3D WebGIS video fusion method under weak constraints according to claim 1, characterized in that: The optimization and adjustment of the 3D model includes the following steps: S2.1 Optimize the surface of the constructed 3D model, and remove factors that affect the projection effect in the video area of ​​the 3D model without affecting the overall effect of the model. Step 2.2: Use the Cesiumlab format conversion tool to convert the model file obj format into the 3DTiles format supported by the Cesium engine.

4. The 3D WebGIS video fusion method under weak constraints according to claim 1, characterized in that: The method also includes selecting the open-source Cecium digital earth engine to build a 3D WebGIS virtual geographic environment, loading satellite image data into Cesium as the virtual environment base map, loading 3DTiles format model data into Cesium to form a virtual three-dimensional geographic scene, and adjusting the spatial position of the three-dimensional model in the virtual environment so that the three-dimensional model coincides with the real geographical location in the satellite image.

5. The 3D WebGIS video fusion method under weak constraints according to claim 1, characterized in that: The method also includes supporting access to local video files and HTTP protocol network video data, and utilizing HTML5 in a Web environment. <video> The tag reads local video data.< / video> 6. The 3D WebGIS video fusion method under weak constraints according to claim 1, characterized in that: The method further includes calculating texture coordinates through projected texture mapping in the GPU fragment shader, including the following steps: S7.1, transform the vertices of the 3D model from the model coordinate system to the camera coordinate system through the model-view matrix; S7.2, transform to world coordinate system using the camera view inverse matrix; S 7.3, transform it to the projection coordinate system according to the projection view matrix; S 7.4, transform to the clipping coordinate system using the projection matrix; S 7.5, normalize the obtained projection vertex coordinates to the [0, 1] interval to obtain the projection texture coordinates.

7. The 3D WebGIS video fusion method under weak constraints according to claim 1, characterized in that: The method further includes fusing the static texture of the 3D model surface with the dynamic video texture, making the dynamic video texture the top layer texture, multiplying the camera view matrix and the inverse view projection matrix in the virtual scene to obtain the cube model view, and similarly obtaining the inverse cube model view, and assigning the two model views to the consistency mapping parameters of the classification primitive class to realize the projection of video images into the 3D scene.

Images