Digital twin video holographic display method and system based on real geographic space

Abstract

The invention discloses a digital twin video holographic display method and system based on a real geographic space, and the method comprises the steps: reading a real-time image frame at a current moment from real-time video data, and obtaining a real-time image frame according to the visibility and shielding relation between the real-time image frame and a three-dimensional live-action model; the method comprises the following steps: dividing an image texture of a real-time image frame into a texture mapping overlapping region and a non-overlapping region, for the texture mapping overlapping region, combining a plurality of historical image frames as a texture source, and adopting a multi-source texture fusion algorithm based on real geographic space consistency to reconstruct a texture map of the texture mapping overlapping region; dynamically mapping the texture map to the surface of the three-dimensional live-action model to obtain video-model data after texture mapping; and transmitting video-model data into a holographic display system, and displaying a holographic image according to the position difference of the eyes of the observer in the space. According to the invention, accurate space fusion is carried out on the three-dimensional real scene model and the real-time video data, and real, real-time and spatially synchronized three-dimensional holographic presentation of a physical scene is realized.

Description

Technical Field

[0001] This invention relates to the field of holographic display technology, specifically to a digital twin video holographic display method and system based on real geographic space. Background Technology

[0002] With the development of digital twin technology, 3D modeling-based digital twins have been widely applied in scenarios such as urban management, industrial monitoring, and simulation training. Traditional digital twin systems typically use static texture maps to construct virtual scenes, which cannot reflect the dynamic changes of the physical world in real time, resulting in a limited display effect and a lack of real-time responsiveness and interactivity. Existing video display methods mainly use 2D displays for playback, failing to provide users with a immersive, stereoscopic experience. Furthermore, textures in traditional 3D modeling are usually derived from photographs, lacking correlation with real-time changing geographic information, thus failing to achieve accurate visualization of real-time physical space. Summary of the Invention

[0003] The purpose of this invention is to provide a digital twin video holographic display method and system based on real geographic space, which can accurately integrate three-dimensional real scene models with real-time video data to achieve a real, real-time, and spatially synchronized three-dimensional holographic presentation of physical scenes.

[0004] To achieve the above objectives, this application proposes the following technical solution: On the one hand, the present invention provides a method for digital twin video holographic display based on real geographic space, specifically including the following steps: S1. Obtain a 3D real-scene model of the target area and acquire real-time video data of the target area from the video acquisition device; S2. Read the real-time image frame at the current moment from the real-time video data. Based on the visibility and occlusion relationship between the real-time image frame and the 3D real scene model, divide the image texture of the real-time image frame into texture mapping overlapping areas and non-overlapping areas. For texture mapping overlap areas, multiple historical image frames are combined as texture sources, and a multi-source texture fusion algorithm based on real geospatial consistency is adopted to reconstruct the texture map of the texture mapping overlap area and dynamically map the texture map onto the surface of the 3D real scene model. S3. After mapping all overlapping and non-overlapping areas of the texture mapping onto the 3D real-world model, the texture-mapped video-model data is obtained. S4. Input the video-model data into the holographic display system and display the holographic image according to the difference in the position of the observer's eyes in space.

[0005] In some specific implementation schemes, the process of determining the visibility and occlusion relationship between real-time image frames and 3D reality models is as follows: S01. Project each pixel in the real-time image frame onto the view frustum space of the camera coordinate system through the camera intrinsic parameter matrix of the video acquisition device, and then combine it with the depth information to back-project it onto the three-dimensional space to obtain the three-dimensional point. S02. Based on the six-degree-of-freedom spatial pose information of the video acquisition device, convert the three-dimensional points into spatial points in the coordinate system of the three-dimensional real scene model; S03. Based on the spatial point coordinates, find the triangular facets corresponding to the surface of the 3D real scene model and store them in the triangular facet set. Combine the reprojection method of real geographic aperture projection geometry to determine the visibility and occlusion relationship between the pixels of the real-time image frame and the triangular facets.

[0006] In some specific implementation schemes, the reprojection method proceeds as follows: For each pixel coordinate, construct a unit view vector. Within the set back projection depth range, determine whether the unit view vector intersects with any triangle in the triangle patch set. If it exists, it means there is no occlusion, the pixel is visible, and the texture area corresponding to the pixel is a non-overlapping area; If it does not exist, it means that there is an intermediate occlusion, and the texture area corresponding to this pixel is a texture overlap area.

[0007] In some specific implementation schemes, the specific process of the multi-source texture fusion algorithm is as follows: S21. Obtain the texture information of N-1 historical image frames before the current time and the current texture state of the 3D real scene model. Use the N-1 frame texture information and the current texture state as texture sources to obtain N texture sources. S22. Calculate the texture difference between the image texture of the real-time image frame and N texture sources respectively, obtain the fusion weight in each texture source in the texture mapping overlap area, and calculate the fused texture value in the texture mapping overlap area to obtain the real-time updated texture map; S23. Dynamically map the real-time updated texture map onto the surface of the 3D real-world model.

[0008] In some specific implementations, the fusion weight in step S22 includes multiple fusion factors, and the calculation method for the fusion weight is as follows: Obtain the angle θ between the surface normal vector of a pixel (x, y) in the overlapping region of the texture map and the viewpoint of the i-th texture source. i Calculate the direction and angle; Get the brightness value of pixel (x, y) under the i-th texture source. L i (x, y); and based on the brightness values ​​of N texture sources, calculate the average illuminance of the pixel under all texture sources. and differences in illuminance; Calculate the straight-line distance from the video acquisition device to the 3D point. d i (x, y); When the unit view vector of a pixel intersects with multiple triangular facets, calculate the intersection distance between the unit view vector and each triangular facet, and select the shortest intersection distance as denoted as . The occlusion depth difference is calculated based on the straight-line distance and the shortest intersection distance; The line-of-sight ratio is calculated based on the proportion of unobstructed segments in the path traversed by a unit line-of-sight vector. S i (x, y); After assigning different weight parameters to the direction angle, illumination difference, occlusion depth difference, and line-of-sight ratio, the data is fused to obtain the fusion weight. W i (x, y): in, This represents the weighting parameter.

[0009] In some specific implementations, texture values ​​are merged. Blending weights for each texture source W i (x, y) and the original texture values ​​of each texture source T i The weighted average of (x, y) is calculated as follows: .

[0010] In some specific implementations, step S22 further includes an iterative optimization step, whereby after obtaining the fused texture value, residual analysis is performed between the fused texture value and the original texture value of each texture source: ; Determine whether the residual size R(x,y) meets the set residual threshold. If not, adjust the weight parameters of each factor in the fusion weighting, recalculate the fusion texture value, and continue iteratively until the residual size meets the set residual threshold.

[0011] In some specific implementations, for non-overlapping regions, the pixels in the real-time image frame are mapped to the patch texture of the corresponding texture region, completing the single-frame texture mapping and replacement operation.

[0012] In some specific implementation schemes, the specific process of constructing the holographic image in step S4 is as follows: The system acquires the observer's spatial position and observation point on the 3D reality model in real time, and calculates the observer's gaze direction vector. By mapping the gaze direction vector to the video source orientation of the video acquisition device using coordinate differences, the difference in the video deflection angle that needs to be adjusted is obtained. Adjust the angle offset of the video acquisition device according to the difference in video deflection angle so that the observer's viewpoint is in the same direction as the 3D reality model; The system obtains the positional differences of the observer's two eyes in space, acquires the projection directions corresponding to the left and right eye perspectives of the observer, generates left and right views that conform to the stereoscopic vision difference of the human eye based on video-model data, and displays holographic images to the observer through a holographic display system.

[0013] Secondly, this application provides a digital twin video holographic display system based on real geographic space, comprising: The data acquisition module is used to acquire a 3D real-scene model of the target area and to acquire real-time video data of the target area from the video acquisition device. The image mapping module is used to read the real-time image frame at the current moment from the real-time video data. Based on the visibility and occlusion relationship between the real-time image frame and the 3D reality model, the image texture of the real-time image frame is divided into texture mapping overlapping area and non-overlapping area. For the texture mapping overlapping area, a multi-source texture fusion algorithm based on the consistency of real geospatial space is used to reconstruct the texture map of the texture mapping overlapping area and dynamically map the texture map onto the surface of the 3D reality model. The image fusion module is used to map all overlapping and non-overlapping areas of the texture mapping onto the 3D real-world model to obtain texture-mapped video-model data. Holographic display systems are used to display holographic images based on video-model data and the spatial differences in the positions of the observer's eyes.

[0014] The beneficial effects of this invention are as follows: This invention proposes a digital twin video holographic display technology for real geographic space, which realizes accurate spatial fusion of real-scene models and real-time video data. By combining the textures of historical multi-frame images and the textures of the 3D real-scene model as texture sources, a multi-source texture fusion algorithm is implemented to make texture mapping more accurate and the stereoscopic effect better. This achieves a realistic, real-time, and spatially synchronized 3D holographic presentation of the physical scene, providing observers with an immersive and interactive experience. Attached Figure Description

[0015] Figure 1 A flowchart of a digital twin video holographic display method based on real geographic space provided in an embodiment of the present invention; Figure 2 This is a flowchart of a multi-source texture fusion algorithm provided in an embodiment of the present invention; Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0018] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0019] Furthermore, for clarity and brevity, descriptions of well-known structures, functions, and configurations may have been omitted. Those skilled in the art will recognize that various changes and modifications can be made to the examples described herein without departing from the spirit and scope of this disclosure.

[0020] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0021] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0022] Example 1 like Figure 1 As shown, this embodiment provides a method for digital twin video holographic display based on real geographic space, specifically including the following steps: S1. Obtain a 3D real-scene model of the target area and acquire real-time video data of the target area from the video acquisition device; Step S1 primarily involves the acquisition of display information, including a 3D reality model and real-time video data. The 3D reality model can be obtained using an oblique photogrammetry system or a mobile surveying device to acquire a 3D reality model of the target area. The 3D reality model established in this application adopts a global rectangular coordinate system (such as the WGS-84 projected coordinate system), and a mapping relationship is established between the model coordinate system and the geographic coordinate system to achieve the correspondence between geospatial information and the 3D reality model. Real-time video data is acquired through video capture devices deployed in the target area. High-precision positioning sensor technology is used to obtain the six-degree-of-freedom spatial pose information of the video capture devices in real time, including their spatial position. and attitude angle ,in Let x, y, and z be the rotation matrices about the x, y, and z axes, respectively. The position of the camera in the 3D model coordinate system can be represented by the rigid body transformation matrix. ; S2. Read the real-time image frame at the current moment from the real-time video data. Based on the visibility and occlusion relationship between the real-time image frame and the 3D real scene model, divide the image texture of the real-time image frame into texture mapping overlapping areas and non-overlapping areas. The specific process for determining the visibility and occlusion relationship between real-time image frames and 3D reality models is as follows: S01. Project each pixel in the real-time image frame onto the view frustum space of the camera coordinate system through the camera intrinsic parameter matrix of the video acquisition device, and then combine it with the depth information to back-project it onto the three-dimensional space to obtain the three-dimensional point. Specifically, the real-time image frames acquired by the video capture device Each pixel (x, y) is projected onto the view frustum space in the camera coordinate system through the camera intrinsic parameter matrix K, and then combined with the depth information d(x, y) for backprojection into 3D space. This projection process is as follows: Where K is the camera intrinsic parameter matrix: S02. Based on the six-degree-of-freedom spatial pose information of the video acquisition device, convert the three-dimensional points into spatial points in the coordinate system of the three-dimensional real scene model; Six-DOF spatial pose information, as an external parameter, includes spatial position. and attitude angle ; the obtained three-dimensional points P cam The points are calculated by transforming them to the 3D model coordinate system using extrinsic parameters, as follows: , This spatial point P world Coordinates are used to find the corresponding triangular facets on the surface of the 3D model to ensure image frame rate. The image texture can be realistically mapped onto the surface of the 3D model. A reprojection method combining real geographic aperture projection geometry is used to determine its visibility and occlusion relationship. S03. Based on the spatial point coordinates, find the triangular facets corresponding to the surface of the 3D real scene model and store them in the triangular facet set. Combine the reprojection method of real geographic aperture projection geometry to determine the visibility and occlusion relationship between the pixels of the real-time image frame and the triangular facets.

[0023] The specific process of the reprojection method is as follows: For each pixel coordinate, construct a unit view vector. Within the set back projection depth range, determine whether the unit view vector intersects with any triangle in the triangle patch set. If it exists, it means there is no occlusion, the pixel is visible, and the texture area corresponding to the pixel is a non-overlapping area; If it does not exist, it means that there is an intermediate occlusion, and the texture area corresponding to this pixel is a texture overlap area.

[0024] The reprojection method can be expressed mathematically as follows: Given pixel coordinates (x, y), construct a unit gaze vector: Where K is the camera intrinsic parameter matrix, and R is the extrinsic parameter rotation matrix corresponding to the attitude angle; let the camera center coordinates be C, and the viewing direction of the pixel be n, then the back projection point of the pixel is: ; Where λ is the back-projection depth, which depends on the intersection of the pixel and the geographic model. Assume the terrain model contains a set of triangular patches. If, within the interval t∈(0,λ), there exists a triangular facet that intersects the line-of-sight ray for any j, t, the judgment function is expressed as: when If the result is positive, it means there is no obstruction and the pixel is visible; otherwise, there is an obstruction in the middle and the view is blocked. The judgment result is passed to the next step to perform texture mapping operation. 1. For non-overlapping regions, the pixels in the real-time image frame are mapped to the corresponding texture area's patch texture, completing the single-frame texture mapping and replacement operation.

[0025] 2. For texture mapping overlap areas, a multi-source texture fusion algorithm based on real geospatial consistency is adopted, which combines multiple historical image frames as texture sources to reconstruct the texture map of the texture mapping overlap area and dynamically map the texture map onto the surface of the 3D real scene model. The multi-source texture fusion algorithm is based on the current texture state of the 3D model and the texture information from N-1 historical frames, utilizing the spatial pixels. Calculate the fusion weights based on the viewing angle, illumination difference, occlusion depth difference, visual bandwidth, and texture confidence among the N texture information sources mentioned above, such as... Figure 2 As shown, the specific process is as follows: S21. Obtain the texture information of N-1 historical image frames before the current time and the current texture state of the 3D real scene model. Use the N-1 frame texture information and the current texture state as texture sources to obtain N texture sources. S22. Calculate the texture difference between the image texture of the real-time image frame and N texture sources respectively, obtain the fusion weight in each texture source in the texture mapping overlap area, and calculate the fused texture value in the texture mapping overlap area to obtain the real-time updated texture map; The fusion weight in step S22 includes multiple fusion factors, and the calculation method for the fusion weight is as follows: Obtain the angle θ between the surface normal vector of a pixel (x, y) in the overlapping region of the texture map and the viewpoint of the i-th texture source. i Calculate the direction and angle; Get the brightness value of pixel (x, y) under the i-th texture source. L i (x, y); and based on the brightness values ​​of N texture sources, calculate the average illuminance of the pixel under all texture sources. and differences in illuminance; Calculate the straight-line distance from the video acquisition device to the 3D point. d i (x, y); When the unit view vector of a pixel intersects with multiple triangular facets, calculate the intersection distance between the unit view vector and each triangular facet, and select the shortest intersection distance as denoted as . The occlusion depth difference is calculated based on the straight-line distance and the shortest intersection distance; The line-of-sight ratio is calculated based on the proportion of unobstructed segments in the path traversed by a unit line-of-sight vector. S i (x, y); After assigning different weight parameters to the direction angle, illumination difference, occlusion depth difference, and line-of-sight ratio, the data is fused to obtain the fusion weight. W i (x, y): in, This represents the weighting parameter.

[0026] Based on the fusion weights, the fused texture value is calculated. Blending weights for each texture source W i (x, y) and the original texture values ​​of each texture source T i The weighted average of (x, y) is calculated as follows: .

[0027] To avoid texture artifacts caused by highly reflective areas, a brightness threshold is set. L max In the texture source L i (x, y) is set as the mean. After texture fusion is complete, an iterative optimization step is also included. After obtaining the fused texture value, residual analysis is performed between the fused texture value and the original texture values ​​of each texture source. Determine if the residual magnitude R(x,y) meets the set residual threshold. If not, adjust the weight parameters of each factor in the fusion weighting process accordingly. The fused texture values ​​are recalculated until the residual size meets the set residual threshold, and then iterative optimization is performed.

[0028] S23. Dynamically map the real-time updated texture map onto the surface of the 3D real-world model. The multi-source texture fusion algorithm proposed in this application makes full use of real geographic spatial orientation, distance and occlusion relationships to improve the continuity and realism of the fused texture, and finally generates a real-time updated texture map, which is dynamically mapped onto the surface of the 3D real-world model; S3. After mapping all overlapping and non-overlapping areas of the texture mapping onto the 3D real-world model, the texture-mapped video-model data is obtained. S4. Input the video-model data into the holographic display system, and display the holographic image based on the difference in spatial position between the observer's two eyes. The specific process is as follows: Real-time acquisition of the observer's spatial location and the observer's viewpoint on the 3D reality model P tar The observation point refers to the location of the human eye's view of the model; for example, it is a specific area within the model being gazed upon, and the observer's gaze direction vector is calculated. : By mapping the gaze direction vector to the video source orientation of the video acquisition device using coordinate differences, the difference in the video deflection angle that needs to be adjusted is obtained. Adjust the angle offset of the video acquisition device based on the difference in video deflection angle. Then, the observer's perspective is aligned with the 3D reality model to ensure consistency between the observer's perspective and the real geographic information. The system obtains the positional differences of the observer's two eyes in space, acquires the projection directions corresponding to the left and right eye perspectives of the observer, generates left and right views that conform to the stereoscopic vision difference of the human eye based on video-model data, and reconstructs holographic images with a realistic sense of depth through a holographic display system to achieve a dynamic and stereoscopic three-dimensional digital twin holographic reconstruction effect.

[0029] Example 2 This embodiment provides a digital twin video holographic display system based on real geographic space, comprising: The data acquisition module is used to acquire a 3D real-scene model of the target area and to acquire real-time video data of the target area from the video acquisition device. The image mapping module is used to read the real-time image frame at the current moment from the real-time video data. Based on the visibility and occlusion relationship between the real-time image frame and the 3D reality model, the image texture of the real-time image frame is divided into texture mapping overlapping area and non-overlapping area. For the texture mapping overlapping area, a multi-source texture fusion algorithm based on the consistency of real geospatial space is used to reconstruct the texture map of the texture mapping overlapping area and dynamically map the texture map onto the surface of the 3D reality model. The image fusion module is used to map all overlapping and non-overlapping areas of the texture mapping onto the 3D real-world model to obtain texture-mapped video-model data. Holographic display systems are used to display holographic images based on video-model data and the spatial differences in the positions of the observer's eyes.

[0030] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for digital twin video holographic display based on real geographic space, characterized in that, Specifically, the following steps are included: S1. Obtain a 3D real-scene model of the target area and acquire real-time video data of the target area from the video acquisition device; S2. Read the real-time image frame at the current moment from the real-time video data. Based on the visibility and occlusion relationship between the real-time image frame and the 3D real scene model, divide the image texture of the real-time image frame into texture mapping overlapping areas and non-overlapping areas. For texture mapping overlap areas, multiple historical image frames are combined as texture sources, and a multi-source texture fusion algorithm based on real geospatial consistency is adopted to reconstruct the texture map of the texture mapping overlap area and dynamically map the texture map onto the surface of the 3D real scene model. S3. After mapping all overlapping and non-overlapping areas of the texture mapping onto the 3D real-world model, the texture-mapped video-model data is obtained. S4. Input the video-model data into the holographic display system and display the holographic image according to the difference in the position of the observer's eyes in space.

2. The method for digital twin video holographic display based on real geographic space according to claim 1, characterized in that, The specific process for determining the visibility and occlusion relationship between real-time image frames and 3D reality models is as follows: S01. Project each pixel in the real-time image frame onto the view frustum space of the camera coordinate system through the camera intrinsic parameter matrix of the video acquisition device, and then combine it with the depth information to back-project it onto the three-dimensional space to obtain the three-dimensional point. S02. Based on the six-degree-of-freedom spatial pose information of the video acquisition device, convert the three-dimensional points into spatial points in the coordinate system of the three-dimensional real scene model; S03. Based on the spatial point coordinates, find the triangular facets corresponding to the surface of the 3D real scene model and store them in the triangular facet set. Combine the reprojection method of real geographic aperture projection geometry to determine the visibility and occlusion relationship between the pixels of the real-time image frame and the triangular facets.

3. The method for digital twin video holographic display based on real geographic space according to claim 2, characterized in that, The specific process of the reprojection method is as follows: For each pixel coordinate, construct a unit view vector. Within the set back projection depth range, determine whether the unit view vector intersects with any triangle in the triangle patch set. If it exists, it means there is no occlusion, the pixel is visible, and the texture area corresponding to the pixel is a non-overlapping area; If it does not exist, it means that there is an intermediate occlusion, and the texture area corresponding to this pixel is a texture overlap area.

4. The method for digital twin video holographic display based on real geographic space according to claim 3, characterized in that, The specific process of the multi-source texture fusion algorithm is as follows: S21. Obtain the texture information of N-1 historical image frames before the current time and the current texture state of the 3D real scene model. Use the N-1 frame texture information and the current texture state as texture sources to obtain N texture sources. S22. Calculate the texture difference between the image texture of the real-time image frame and N texture sources respectively, obtain the fusion weight in each texture source in the texture mapping overlap area, and calculate the fused texture value in the texture mapping overlap area to obtain the real-time updated texture map; S23. Dynamically map the real-time updated texture map onto the surface of the 3D real-world model.

5. The method for digital twin video holographic display based on real geographic space according to claim 4, characterized in that, The fusion weight in step S22 includes multiple fusion factors, and the calculation method for the fusion weight is as follows: Obtain the angle θ between the surface normal vector of a pixel (x, y) in the overlapping region of the texture map and the viewpoint of the i-th texture source. i Calculate the direction and angle; Get the brightness value of pixel (x, y) under the i-th texture source. L i (x, y); and based on the brightness values ​​of N texture sources, calculate the average illuminance of the pixel under all texture sources. and differences in illuminance; Calculate the straight-line distance from the video acquisition device to the 3D point. d i (x, y); When the unit view vector of a pixel intersects with multiple triangular facets, calculate the intersection distance between the unit view vector and each triangular facet, and select the shortest intersection distance as denoted as . The occlusion depth difference is calculated based on the straight-line distance and the shortest intersection distance; The line-of-sight ratio is calculated based on the proportion of unobstructed segments in the path traversed by a unit line-of-sight vector. S i (x, y); After assigning different weight parameters to the direction angle, illumination difference, occlusion depth difference, and line-of-sight ratio, the data is fused to obtain the fusion weight. W i (x, y): in, This represents the weighting parameter.

6. The method for digital twin video holographic display based on real geographic space according to claim 4, characterized in that, Blend texture values Blending weights for each texture source W i (x, y) and the original texture values ​​of each texture source T i The weighted average of (x, y) is calculated as follows: 。 7. The method for digital twin video holographic display based on real geographic space according to claim 6, characterized in that, Step S22 also includes an iterative optimization step, after obtaining the fused texture value, performing residual analysis between the fused texture value and the original texture value of each texture source: Determine whether the residual size R(x,y) meets the set residual threshold. If not, adjust the weight parameters of each factor in the fusion weighting, recalculate the fusion texture value, and continue iteratively until the residual size meets the set residual threshold.

8. The method for digital twin video holographic display based on real geographic space according to claim 3, characterized in that, For non-overlapping regions, the pixels in the real-time image frame are mapped to the corresponding texture area's patch texture, completing the single-frame texture mapping and replacement operation.

9. The method for digital twin video holographic display based on real geographic space according to claim 1, characterized in that, The specific process of constructing the holographic image in step S4 is as follows: The system acquires the observer's spatial position and observation point on the 3D reality model in real time, and calculates the observer's gaze direction vector. By mapping the gaze direction vector to the video source orientation of the video acquisition device using coordinate differences, the difference in the video deflection angle that needs to be adjusted is obtained. Adjust the angle offset of the video acquisition device according to the difference in video deflection angle so that the observer's viewpoint is in the same direction as the 3D reality model; The system obtains the positional differences of the observer's two eyes in space, acquires the projection directions corresponding to the left and right eye perspectives of the observer, generates left and right views that conform to the stereoscopic vision difference of the human eye based on video-model data, and displays holographic images to the observer through a holographic display system.

10. A digital twin video holographic display system based on real geographic space, characterized in that, include: The data acquisition module is used to acquire a 3D real-scene model of the target area and to acquire real-time video data of the target area from the video acquisition device. The image mapping module is used to read the real-time image frame at the current moment from the real-time video data. Based on the visibility and occlusion relationship between the real-time image frame and the 3D reality model, the image texture of the real-time image frame is divided into texture mapping overlapping area and non-overlapping area. For the texture mapping overlapping area, a multi-source texture fusion algorithm based on the consistency of real geospatial space is used to reconstruct the texture map of the texture mapping overlapping area and dynamically map the texture map onto the surface of the 3D reality model. The image fusion module is used to map all overlapping and non-overlapping areas of the texture mapping onto the 3D real-world model to obtain texture-mapped video-model data. Holographic display systems are used to display holographic images based on video-model data and the spatial differences in the positions of the observer's eyes.

Images