A method and system for interactive display of tourism projects based on three-dimensional modeling and augmented reality

Abstract

The application relates to the field of three-dimensional modeling technology and discloses a tourism project interactive display method and system based on three-dimensional modeling and augmented reality, wherein the tourism project interactive display method based on three-dimensional modeling and augmented reality comprises the following steps: step S1, calculating initial pixel points; step S2, calculating comprehensive reflection coefficients; step S3, calculating an image gradient matrix; step S4, calculating a parallax folding coefficient; step S5, calculating a modified rotation matrix; step S6, generating a virtual image; and step S7, generating an augmented reality image. The parallax folding coefficient can comprehensively evaluate the reflection misregistration risk of each piece element near a tide line, the weighted pose solving, the modulation of the depth of a virtual image and the virtual brightness can simultaneously solve the problems of false hard occlusion caused by a wet film virtual image and inconsistent illumination, the interactive operation is prevented from being disturbed by the wet film virtual image, and the interactive experience accuracy of tourists is improved.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and more specifically, to a method and system for interactive display of tourism projects based on 3D modeling and augmented reality. Background Technology

[0002] Augmented reality (AR) technology has been widely applied in the field of cultural heritage tourism guidance, enabling the overlay of computer-generated 3D reconstruction models onto real-world scenes to achieve visual restoration and interactive experiences of historical sites. Existing technologies primarily focus on outdoor AR (Augmented Reality) positioning and registration based on VSLAM (Visual Simultaneous Localization and Mapping), illumination consistency optimization based on multi-source geographic information, and improvements in 3D imaging for highly reflective surfaces. These technologies estimate camera pose by extracting image feature points, adjust the brightness of virtual models by matching real-world lighting parameters, or employ hardware methods to reduce the interference of reflections on depth perception, achieving basic AR display functions in most typical outdoor scenarios.

[0003] In unique settings like coastal tidal cultural heritage sites, existing technologies prove insufficient. Periodic tidal changes create a thin water film on the stone surface. Combined with variations in the sun's angle of incidence and the periodic texture of the stone structures, this results in both solid stone textures and the virtual texture of the water film appearing simultaneously near the tide line. This phenomenon can lead to feature mismatches during image registration, causing virtual models to float and become misaligned. During depth perception, the virtual image may be mistaken for a real surface, resulting in incorrect occlusion of virtual components. Furthermore, during interaction, the landing point may be mapped to the virtual image layer, leading to missed selections or triggering incorrect components.

[0004] The root cause of these shortcomings lies in the fact that existing technologies typically treat reflections, highlights, and water surface changes as random noise or outliers and uniformly eliminate them without considering the specific physical characteristics of coastal tidal scenes. This approach results in the loss of a large amount of real, usable feature information near the tide line and fails to distinguish between real textures and reflected virtual images, making it difficult to fundamentally solve the end-to-end error problem caused by pseudo-double-layer features. Therefore, it is necessary to develop an AR interactive display method adapted to coastal tidal heritage scenes to improve the stability and interactive accuracy of AR displays in such scenarios. Summary of the Invention

[0005] This invention provides a method and system for interactive display of tourism projects based on 3D modeling and augmented reality, solving the technical problems mentioned in the background.

[0006] This invention provides a method for interactive display of tourism projects based on 3D modeling and augmented reality, comprising the following steps: Step S1: Obtain the current image, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and 3D model. Combine the intrinsic parameter matrix, initial rotation matrix, and initial translation vector to perform coordinate transformation, project the fragment center point of the 3D model onto the current image, and obtain the initial pixel point. Step S2: Calculate the composite reflection coefficient based on fragment elevation, fragment normal vector, tide level, line-of-sight vector, and illumination vector. Step S3: Match the initial pixel points, the current image, and the model texture template to calculate the image gradient matrix and the corresponding pixel points of the fragments; Step S4: Calculate the parallax folding coefficient based on the fragment elevation, tide level, initial pixel, tide line tangential vector, image gradient matrix, and comprehensive reflection coefficient. Step S5: Calculate the pose solution weights based on the disparity folding coefficients, and combine the pose solution weights with the corresponding pixel points of the fragments to calculate the corrected rotation matrix and the corrected translation vector. Step S6: Calculate the occlusion depth and model radiance based on the corrected rotation matrix, corrected translation vector, disparity folding coefficient, and integrated reflection coefficient to generate a virtual image; Step S7: Obtain interactive pixels and mask matrix. Calculate interactive 3D coordinates based on interactive pixels, corrected rotation matrix, corrected translation vector, and 3D model. Combine the mask matrix with the current image and virtual image to generate an augmented reality image.

[0007] This invention provides an interactive display system for tourism projects based on 3D modeling and augmented reality, comprising: The fragment center projection module acquires the current image, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and 3D model. It then performs coordinate transformation by combining the intrinsic parameter matrix, initial rotation matrix, and initial translation vector to project the fragment center point of the 3D model onto the current image, thereby obtaining the initial pixel point. The comprehensive reflection coefficient calculation module calculates the comprehensive reflection coefficient based on the fragment elevation, fragment normal vector, tide height, line-of-sight vector, and illumination vector. The texture matching module matches the initial pixel points, the current image, and the model texture template to calculate the image gradient matrix and the corresponding pixel points of the fragments. The parallax folding coefficient calculation module calculates the parallax folding coefficient based on the fragment elevation, tide height, initial pixel, tide line tangential vector, image gradient matrix, and comprehensive reflection coefficient. The pose solving module calculates the pose solving weights based on the disparity folding coefficients, and combines the pose solving weights with the corresponding pixel points of the fragments to calculate the corrected rotation matrix and the corrected translation vector. The virtual image generation module calculates the occlusion depth and model radiance based on the corrected rotation matrix, corrected translation vector, disparity folding coefficient, and comprehensive reflectance coefficient to generate a virtual image. The augmented reality image generation module obtains interactive pixels and a mask matrix. Based on the interactive pixels, the corrected rotation matrix, the corrected translation vector, and the 3D model, it calculates the interactive 3D coordinates. Then, it combines the mask matrix with the current image and the virtual image to generate an augmented reality image.

[0008] The beneficial effects of this invention are as follows: By employing a method that calculates the parallax folding coefficient and uses it as the core driving variable, this invention can comprehensively assess the risk of misregistration of reflections in each fragment near the tide line, providing a unified basis for subsequent end-to-end error compensation. By adopting a weighted pose solving method based on this feature, the influence of wet film virtual images on camera pose can be reduced from the registration source, which helps improve the positioning stability of the virtual model. By using the same feature to simultaneously modulate occlusion depth and virtual brightness, the problem of erroneous hard occlusion caused by wet film virtual images and the problem of inconsistent lighting can be solved simultaneously, which helps improve the fusion effect of virtual and real content. By adopting a method of calculating interactive landing points by correcting pose, the interference of water film virtual images on interactive operations can be avoided, which helps improve the accuracy of the visitor's interactive experience. Attached Figure Description

[0009] Figure 1 This is a calculation flowchart of a tourism project interactive display method based on 3D modeling and augmented reality according to the present invention; Figure 2 This is a schematic diagram of a module of an interactive display system for tourism projects based on 3D modeling and augmented reality according to the present invention. Detailed Implementation

[0010] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0011] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" indicate that the element or object preceding the term encompasses the elements or objects listed following the term and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0012] like Figures 1-2 As shown, an interactive display method for tourism projects based on 3D modeling and augmented reality includes the following steps: Step S1: Obtain the current image, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and 3D model. Combine the intrinsic parameter matrix, initial rotation matrix, and initial translation vector to perform coordinate transformation, project the fragment center point of the 3D model onto the current image, and obtain the initial pixel point. Step S2: Calculate the composite reflection coefficient based on fragment elevation, fragment normal vector, tide level, line-of-sight vector, and illumination vector. Step S3: Match the initial pixel points, the current image, and the model texture template to calculate the image gradient matrix and the corresponding pixel points of the fragments; Step S4: Calculate the parallax folding coefficient based on the fragment elevation, tide level, initial pixel, tide line tangential vector, image gradient matrix, and comprehensive reflection coefficient. Step S5: Calculate the pose solution weights based on the disparity folding coefficients, and combine the pose solution weights with the corresponding pixel points of the fragments to calculate the corrected rotation matrix and the corrected translation vector. Step S6: Calculate the occlusion depth and model radiance based on the corrected rotation matrix, corrected translation vector, disparity folding coefficient, and integrated reflection coefficient to generate a virtual image; Step S7: Obtain interactive pixels and mask matrix. Calculate interactive 3D coordinates based on interactive pixels, corrected rotation matrix, corrected translation vector, and 3D model. Combine the mask matrix with the current image and virtual image to generate an augmented reality image.

[0013] In one embodiment of the present invention, the specific calculation process of step S1 includes: Based on the intrinsic parameter matrix, initial rotation matrix, initial translation vector, and fragment center point, the formulas for calculating the initial pixel point and initial depth value are as follows: ; ; in, Indicates the first Each element at any time The initial pixel point, Indicates the first Each element at any time The initial depth value, Represents the third in the three-dimensional model Each piece costs 1 yuan. Indicates the current moment. Represents the intrinsic parameter matrix, Indicates time The initial rotation matrix, Indicates time The initial translation vector, Indicates the first The center point of each element, This represents the pinhole projection operator. This represents the camera depth axis vector.

[0014] It should be noted that a 3D model fragment is the smallest processing unit after the surface of a 3D model has been discretized. Each fragment contains a center point, a normal vector, and a texture template. The camera intrinsic matrix is ​​a matrix containing imaging parameters such as camera focal length and principal point, obtained from the camera's factory calibration. The initial rotation matrix is ​​the rotation parameter that converts world coordinates to camera coordinates. The initial translation vector is the translation parameter that converts world coordinates to camera coordinates. The pinhole projection operator is a geometric transformation operator that maps 3D camera coordinate points to 2D pixel coordinates. The camera depth axis vector is the unit vector representing the depth direction in the camera coordinate system. The initial pixel represents the pixel coordinates of the 3D model fragment projected onto the current image based on its initial pose. The initial depth value represents the depth distance of the 3D model fragment in the camera coordinate system calculated based on its initial pose.

[0015] Specifically, 3D model fragment partitioning can be achieved using three methods: mesh triangular facets, point cloud neighborhood, or neural rendering fragments. The choice depends on the model's accuracy and the device's computing power (custom settings are only briefly described in this invention and will not be elaborated upon here). For example, when the device's computing power is low, mesh triangular facet partitioning can be used, treating each triangular facet of the 3D model as an independent fragment. This method has low computational cost and is suitable for running on mobile devices. For example, when higher model accuracy is required, point cloud neighborhood partitioning can be used, grouping each point in the 3D point cloud and several of its neighboring points into a fragment. This method can retain more model details and is suitable for running on high-performance AR glasses. When real-time neural rendering is required, neural rendering fragment partitioning can be used, dividing the 3D model into multiple neural rendering fragments. Each fragment contains parameters such as color, density, and neural network weights, generated by uniformly sampling the model surface. The sampling density is adjusted according to the model's complexity.

[0016] Specifically, the initial pose can be obtained either through VSLAM system initialization or by pre-calibrating the scenic area's geographic coordinates. During VSLAM system initialization, ORB feature points are extracted and matched from continuously acquired multiple frames of images. The initial camera pose is estimated using epipolar geometric constraints. This method requires no pre-calibration and is suitable for AR displays in unknown scenes. Pre-calibrated scenic area geographic coordinates can be obtained by acquiring the coordinates of multiple feature points within the scenic area using RTK (Real-Time Kinematic) positioning equipment. This establishes a correspondence between the world coordinate system and the actual geographic coordinates of the scenic area. When a visitor enters the scenic area, the initial position is obtained through the device's GPS positioning. The initial camera pose is then calculated by combining this with the pre-calibrated feature point information. This method offers fast initialization speed and high positioning accuracy, making it suitable for AR displays in known scenes. In practical applications, the two methods can be combined. The initial pose can be quickly obtained using pre-calibrated geographic coordinates at system startup. Once the VSLAM system tracks a sufficient number of stable feature points, the system switches to VSLAM for real-time pose correction, balancing initialization speed and positioning accuracy.

[0017] In one embodiment of the present invention, the specific calculation process of step S2 includes: The formula for calculating the elevation of a fragment, based on its center point and local vertical vector, is as follows: ; in, Indicates the first Individual elevation, Represents a local vertical vector. Indicates the first The center point of each element; The formula for calculating the amount of moisture at the tidal line, based on the slab elevation, tidal height, and tidal line dimensions, is as follows: ; in, Indicates the first Each element at any time The amount of moisture at the tide line. Indicates time The height of the tide. Indicates the tidal line scale value; Based on the initial rotation matrix, initial translation vector, fragment center point, fragment normal vector, and lighting vector, the formulas for calculating the view vector, normal coordinate vector, and lighting coordinate vector are as follows:

[0018] ; ; ; in, Indicates the first Each element at any time The line-of-sight vector, Indicates the first Each element at any time The normal coordinate vector, Indicates time The illumination coordinate vector, Indicates the first Each fragment normal vector Indicates time The illumination vector; The formula for calculating the half-angle vector is as follows, based on the line-of-sight vector and the lighting coordinate vector: ;in, Indicates the first Each element at any time A half-angle vector; The formula for calculating the reflection angle coefficient based on the normal coordinate vector, the line-of-sight vector, and the reference reflection coefficient is as follows: ; in, Indicates the first Each element at any time The reflection angle coefficient, Indicates the reference reflection coefficient; The formula for calculating the composite reflection coefficient is as follows, based on the tidal line wetting amount, reflection angle coefficient, normal coordinate vector, and half-angle vector: ; in, Indicates the first Each element at any time The overall reflection coefficient.

[0019] It should be noted that the local vertical vector is a unit vector defining the local elevation direction of the scenic area, calculated by the scenic area's digital elevation model. The tidal level is the height of the current tidal water or water film plane, which can be obtained in real time through tidal level sensors deployed within the scenic area. These sensors are typically deployed near the docks or tidal channels, with a sampling frequency preferably of 1 Hz to reflect real-time tidal changes. The tidal line scale value is a custom parameter controlling the rate of decrease in humidity with elevation difference, preferably 0.2 meters, with a range of 0.1 to 0.5 meters. This value can be adjusted according to the actual tidal range of the coastal site; the larger the tidal range, the larger the value. The fragment normal vector is a unit vector describing the orientation of the fragment surface, calculated from the geometric information of the 3D model. The illumination vector is a unit vector representing the direction of solar incidence, calculated using astronomical algorithms based on the current time and the scenic area's geographical location. The calculation accuracy meets the requirements for consistent outdoor AR illumination. The reference reflectance coefficient is the basic reflectivity of the medium surface when light is incident normally; it is a preset parameter, pre-calibrated based on the stone material of the site. The comprehensive reflection coefficient comprehensively expresses the degree of proximity of the fragment to the tide line, the reflection trend, and whether the reflection direction is towards the observer. Tidal line wetting represents the degree of wetting of the fragment near the tide line. The line-of-sight vector represents the unit direction from the fragment towards the camera. The half-angle vector is used to estimate whether specular highlights are towards the observer. The reflection angle coefficient represents the change in reflection intensity at different incident angles.

[0020] Specifically, when installing tide level sensors, their measurement range should cover the historical highest and lowest tide levels of the scenic area. The installation location should avoid areas with boat traffic and tourist activity to prevent interference. When calculating the illumination vector using astronomical algorithms, the current year, month, day, hour, minute, and second, as well as the longitude and latitude of the scenic area, need to be input. By calculating the sun's declination and hour angle, the azimuth and altitude angles of the sun in the local area are obtained, and then converted into a unit illumination vector in the world coordinate system. The reference reflectance coefficient can be obtained by measuring stone samples from the archaeological site using a spectrophotometer in a laboratory environment. During measurement, a normal incidence method is used to collect reflectance data at different wavelengths, and the average value of the visible light band is taken as the reference reflectance coefficient value.

[0021] Specifically, the tidal line scale value is determined based on the actual tidal range and the distribution characteristics of the humid area at the coastal site. For example, when the tidal range of the site is 2 meters, the preferred tidal line scale value is 0.2 meters. At this value, the rate of decrease in humidity with elevation difference is moderate, accurately reflecting the humidity near the tidal line. When the tidal range of the site is 4 meters, the preferred tidal line scale value is 0.4 meters. At this value, the rate of decrease in humidity is slower, covering a larger area of ​​humidity. In practical applications, the optimal tidal line scale value can be determined by measuring the humid area at different tidal levels in the scenic area and then conducting statistical analysis.

[0022] Specifically, the baseline reflectance coefficient is pre-calibrated based on the stone material of the archaeological site, with different values ​​used for different types of stone. For granite sites, a baseline reflectance coefficient of 0.04 is preferred; for limestone sites, a preferred value of 0.03; and for sandstone sites, a preferred value of 0.05. In practical applications, the baseline reflectance coefficient can be determined by collecting reflectance data of different stones under normal incident conditions and performing statistical analysis. For sites with mixed materials, the baseline reflectance coefficient can be calculated using a weighted average method based on the distribution ratio of different materials.

[0023] It should be noted that the above embodiments transform the natural laws of tides and optical reflection characteristics into continuous weights, rather than simply discarding wet stone surfaces as abnormal areas. This preserves the truly usable feature information near the tide line while suppressing high-risk reflection features, avoiding a decrease in registration accuracy due to feature removal. For example, in the area near the tide line, some fragments, although affected by water film reflection, still contain real stone texture information. Simply removing all these fragments would reduce the number of usable features and decrease the robustness of registration. However, using continuous weights allows adjusting the contribution of fragments in subsequent calculations based on their reflection risk level, preserving truly usable features while suppressing high-risk reflection features. This approach improves the robustness of the system while ensuring registration accuracy.

[0024] In one embodiment of the present invention, the specific calculation process of step S3 includes: The formula for calculating the image gradient matrix based on the current image, the two-dimensional smoothing kernel, and the initial pixels is as follows: ; in, Indicates the first Each element at any time The image gradient matrix, Represents a two-dimensional smoothing kernel. Indicates the smoothness scale. Indicates time The current image, Indicates the first Each element at any time The initial pixel point, Represents the spatial gradient operator. This represents the convolution operation; Based on the initial pixel points, the current image, and the model texture template, the formulas for calculating the candidate offsets and the corresponding pixel points of fragments are as follows: ; ; in, Indicates the first Each element at any time Candidate offsets. This represents the candidate offset in the candidate offset set. Indicates the first A set of candidate offsets for each fragment. This indicates the pixel offset within the template pixel support region. Indicates the first The template pixel support area of ​​each fragment. Indicates the first Each fragment model texture template has a pixel offset. Template value at the location, Indicates the first Mean of texture template for each fragment model Indicates the first Each element at any time and offset The mean of the image patch below, Indicates the stable value of the denominator. Indicates the first Each element at any time The fragment corresponds to the pixel.

[0025] It should be noted that the 2D smoothing kernel is a convolution kernel used for image smoothing. It is a preset parameter, preferably a Gaussian smoothing kernel. The size of the Gaussian smoothing kernel is usually 5×5 pixels, which can effectively suppress salt-and-pepper noise and Gaussian noise in the image while preserving the edge information of the image. The smoothing scale is a parameter that controls the range of the smoothed neighborhood of the image. It is a custom parameter, preferably 3 pixels, with a value ranging from 1 to 5 pixels. It can be adjusted according to the image resolution; the higher the image resolution, the larger the value. The template pixel support area is the pixel range covered by the model texture template. It is a custom parameter, preferably 11×11 pixels, with a value ranging from 7×7 to 15×15 pixels. It can be adjusted according to the fragment size; the larger the fragment size, the larger the value. The candidate offset set is the allowed search range during template matching. It is a custom parameter, preferably ±5 pixels, with a value ranging from ±3 to ±7 pixels. It can be adjusted according to the initial pose accuracy; the lower the initial pose accuracy, the larger the value. The normalized correlation denominator stabilization term is a parameter that prevents the denominator from approaching zero during low-texture region matching calculations. It is a preset parameter, with a preferred value of 0.001. The image gradient matrix represents the rate of change of image brightness in the spatial direction. The candidate offset represents the translation of the model texture template relative to the initial projection position. The pixel corresponding to the fragment is obtained by adding the initial pixel and the candidate offset.

[0026] Specifically, normalized correlation matching determines the optimal matching position by calculating the similarity between the model texture template and the current image patch. The calculation first removes the overall brightness mean of both the template and the image patch, then calculates their cross-correlation value, and finally divides it by the square root of their energy product to obtain the normalized similarity value. The offset corresponding to the maximum similarity value is taken as the optimal matching offset. The candidate offset set is generated using a uniform sampling method. Within the search range around the initial projected pixel, all possible candidate offsets are generated with a step size of 1 pixel to ensure coverage of all possible scenarios for the actual projection position of the fragment.

[0027] Specifically, the template matching search range is limited to a finite region around the initial projected pixels to avoid the inefficiency of full-image searching. For example, when the initial pose accuracy is high, the deviation between the actual projected position and the initial projected position of the fragment is small, and the search range can be limited to ±3 pixels, which can improve calculation speed while ensuring matching accuracy. When the initial pose accuracy is low, the deviation between the actual projected position and the initial projected position of the fragment is large, and the search range can be expanded to ±7 pixels, which can improve the matching success rate. In practical applications, the search range can be dynamically adjusted according to the covariance matrix of the initial pose. When the covariance of the initial pose is large, the search range is expanded; when the covariance of the initial pose is small, the search range is reduced.

[0028] Specifically, the size of the template pixel support area is adjusted according to the fragment size and image resolution to ensure sufficient texture information is included. For example, when the fragment's projection size in the image is 10×10 pixels, the preferred template pixel support area is 11×11 pixels, which can completely cover the fragment's projection area and contain enough texture information for matching; when the fragment's projection size in the image is 14×14 pixels, the preferred template pixel support area is 15×15 pixels, which can also completely cover the fragment's projection area. If the template pixel support area is too small, it will result in insufficient texture information and decreased matching accuracy; if the template pixel support area is too large, it will increase the computational load and reduce the matching speed.

[0029] It should be noted that the above embodiments extract both local gradient directions and template matching results, rather than relying solely on traditional feature points for registration. This allows the use of local gradient directions to determine the reflection amplification effect along the tidal line, while simultaneously utilizing template matching results to improve the robustness of registration. Traditional feature point registration methods only extract salient feature points such as corners and edges in the image. In areas near the tidal line, these feature points are easily affected by water film reflection, leading to matching errors. However, simultaneously extracting local gradient directions and template matching results allows for the comprehensive use of both global and local image information, improving the accuracy and robustness of registration. Local gradient directions reflect the direction of image texture and are used to determine the reflection amplification effect along the tidal line; template matching results provide the precise corresponding position of fragments in the image, used for pose determination.

[0030] In one embodiment of the present invention, the specific calculation process of step S4 includes: The formula for calculating the mirror coordinates of the fragment, based on the fragment center point, fragment elevation, tide level, and local vertical vector, is as follows: ; in, Indicates the first Each element at any time The mirror coordinates of the point. Indicates the first The center point of each element, Indicates the first Individual elevation, Indicates time The height of the tide. Represents a local vertical vector; The formula for calculating the projection difference is as follows, based on the mirror coordinates, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and initial pixel:

[0031] ;in, Indicates the first Each element at any time The projection difference, Represents the intrinsic parameter matrix, Indicates time The initial rotation matrix, Indicates time The initial translation vector, Indicates the first Each element at any time The initial pixel point, Indicates focal length scale, This represents the pinhole projection operator; The formula for calculating the tidal line image vector is as follows, based on the fragment center point, tidal line tangential vector, tidal line length, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and initial pixel point:

[0032] ;in, Indicates the first Each element at any time The tidal line image vector, Indicates the length of the tide line. Represents the tangential vector of the tide line; The formula for calculating the parallax folding coefficient based on the comprehensive reflection coefficient, projection difference, image gradient matrix, and tide line image vector is as follows:

[0033] ;in, Indicates the first Each element at any time The parallax folding factor, Indicates the first Each element at any time The overall reflectance coefficient, Indicates the first Each element at any time The projection difference, Indicates the first Each element at any time The image gradient matrix, Indicates the first Each element at any time The tidal line image vector, This represents the steady-state value of the gradient.

[0034] It should be noted that the mirror coordinate point is the point symmetrical to the fragment center point about the tidal plane, used to simulate the geometric position of the virtual image of the water film. The focal length scale is a camera parameter used to normalize pixel distance; it is a preset parameter determined by the focal length value in the camera's intrinsic parameter matrix. The tidal line tangential vector is a unit vector representing the direction of the tidal line, step edge, or ditch; it is a preset parameter obtained through pre-processing edge detection of the site's 3D terrain. The tidal line length value is a reference length parameter used to calculate the direction of the tidal line image; it is a user-defined parameter, preferably 1 meter, with a range of 0.5 to 2 meters. It can be adjusted according to the actual length of the tidal line; the longer the tidal line, the larger the value. The gradient stability value is a parameter to avoid the denominator being zero during calculation in low-gradient regions; it is a preset parameter, preferably 0.001. The parallax folding coefficient is a continuous image geometry quantity that comprehensively measures the risk of fragment misregistration due to reflection; it represents the degree of misregistration risk caused by tidal line reflection; the larger the value, the higher the risk of misregistration. The tidal line image vector is used to extract the component of the image gradient along the tidal line direction.

[0035] Specifically, when using the Canny edge detection algorithm to extract tidal lines, a dual-threshold method is employed for edge screening. The ratio of the high threshold to the low threshold is typically set to 2:1, with the specific value adjusted based on the resolution and noise level of the site's digital elevation model. This ensures accurate extraction of tidal line edge information while suppressing false edges. Historical tidal line observation data can be obtained through long-term monitoring records from the scenic area management department. The collection period should be no less than one complete tidal cycle, and the collection frequency should be no less than once per hour. By statistically analyzing the historical data, the direction and position of the average tidal line are fitted.

[0036] Specifically, the tidal line tangential vector is generated either by pre-processing edge detection on the 3D topography of the site or by fitting historical tidal line observation data. For example, the Canny edge detection algorithm can be used to process the digital elevation model of the site, extract the edge information of the tidal line, and then perform straight line fitting using the least squares method to obtain the tidal line tangential vector. Alternatively, tidal line observation data from the past year can be collected, and the average tidal line direction can be fitted through statistical analysis as the tidal line tangential vector. For sites with multiple tidal lines, the tangential vector of each tidal line can be extracted separately, and then the corresponding tidal line tangential vector can be selected for calculation based on the location of the fragment.

[0037] Specifically, the mirror point projection difference calculation only applies to fragments whose elevation is close to the tide level; fragments far from the tide line can skip this calculation. For example, when the difference between the fragment's elevation and the tide level is greater than 1 meter, the fragment can be considered unaffected by tide line reflection, and the mirror point projection difference does not need to be calculated. This reduces computational load and improves system speed. For fragments whose elevation difference is less than or equal to 1 meter, the mirror point projection difference needs to be calculated to accurately assess the risk of misregistration due to reflection. In practical applications, this threshold can be adjusted according to the actual conditions of the scenic area. For sites with large tidal ranges, the threshold can be appropriately increased; for sites with small tidal ranges, the threshold can be appropriately decreased.

[0038] In one embodiment of the present invention, the specific calculation process of step S5 includes: The formula for calculating the pose solution weights based on the disparity folding factor is as follows: ; in, Indicates the first Each element at any time The pose calculation weights, Indicates the first Each element at any time The parallax folding factor; Based on the pose-based weights, the corresponding pixels of the fragment, the intrinsic parameter matrix, and the fragment center point, the formulas for calculating the corrected rotation matrix and the corrected translation vector are as follows: ; in, Indicates time The corrected rotation matrix, Indicates time The corrected translation vector, Let represent the rotation matrix to be determined. Let represent the translation vector to be determined. Indicates time The set of projectable elements, Indicates the first Each element at any time The pose calculation weights, Indicates the first Each element at any time The fragment corresponds to the pixel. Represents the intrinsic parameter matrix, Indicates the first The center point of each element, This represents the pinhole projection operator. This represents the variable corresponding to the minimum value of the objective function.

[0039] It should be noted that the pose calculation weights are contribution coefficients of fragments to pose optimization, determined based on reflection risk. These weights represent the importance of fragments in the camera pose calculation process, ranging from 0 to 1; a larger value indicates a greater contribution. The projectable fragment set is the set of fragments currently within the camera's field of view with positive depth. The corrected rotation matrix is ​​the weighted optimized camera rotation matrix. The corrected translation vector is the weighted optimized camera translation vector.

[0040] Specifically, the objective function for minimizing the weighted projection error is composed of the sum of squared weighted projection errors of all projectable fragments. The projection error of each fragment is the Euclidean distance between its actual corresponding pixel in the image and the projected pixel calculated based on the desired pose. This distance is multiplied by the corresponding pose weight, squared, and then summed over all fragments to obtain the objective function value. The initial values ​​of the nonlinear optimization algorithm are the initial rotation matrix and initial translation vector obtained in step one. Iterative optimization is used to minimize the objective function value, resulting in the corrected camera pose.

[0041] Specifically, the set of projectable fragments is obtained through initial depth value filtering, retaining only fragments with high visibility for pose optimization. For example, when the initial depth value is greater than 0 and less than 100 meters, the fragment is considered to be within the camera's field of view and visible, and is added to the set of projectable fragments; when the initial depth value is less than or equal to 0, the fragment is considered to be behind the camera and invisible; when the initial depth value is greater than or equal to 100 meters, the fragment is considered to be too far from the camera, with unclear texture information and minimal contribution to registration, and is not added to the set of projectable fragments. In practical applications, the maximum depth threshold can be adjusted according to the device's performance and the size of the scene. For larger scenes, the maximum depth threshold can be appropriately increased; for smaller scenes, the maximum depth threshold can be appropriately decreased.

[0042] Specifically, the weighted projection error minimization problem is solved using a nonlinear optimization algorithm. The number of iterations is adjusted according to the device's computing power (a custom setting, only briefly described in this invention and not elaborated upon here). For example, when the device's computing power is high, the number of iterations can be set to 20 to obtain a more accurate pose solution; when the device's computing power is low, the number of iterations can be set to 10 to improve computational speed while maintaining a certain level of accuracy. Commonly used nonlinear optimization algorithms include the Levenberg-Marquardt algorithm and the Gauss-Newton algorithm, both of which can effectively solve the weighted projection error minimization problem. In practical applications, an appropriate optimization algorithm can be selected based on the problem's scale and the device's performance.

[0043] It should be noted that the above embodiments directly inject the tidal line reflection features into the pose solving process, rather than performing post-rendering adjustments. This reduces the impact of wet film ghost images on camera pose from the registration source, avoiding image distortion caused by post-processing. Post-processing methods typically adjust the position of the virtual model after rendering to correct registration errors. However, this method can only correct surface visual errors and cannot address errors in camera pose itself, easily leading to phenomena such as shaking and drifting of the virtual model. In contrast, directly injecting the tidal line reflection features into the pose solving process fundamentally corrects camera pose errors, improving the stability and accuracy of registration. This approach allows the position of the virtual model to better match the real scene, enhancing the realism of the AR display.

[0044] In one embodiment of the present invention, the specific calculation process of step S6 includes: The formula for calculating the corrected depth value is as follows, based on the corrected rotation matrix, the corrected translation vector, and the fragment center point: ; in, Indicates the first Each element at any time Corrected depth value, Represents the camera depth axis vector. Indicates time The corrected rotation matrix, Indicates time The corrected translation vector, Indicates the first The center point of each element; The formula for calculating the occlusion depth is as follows, based on the corrected depth value, depth modulation coefficient, and parallax folding coefficient: ; in, Indicates time pixels The depth of occlusion at that location Represents the pixels in the current image. Indicates time Projected onto pixels The set of fragments, Represents the depth modulation coefficient. Indicates the first Each element at any time The parallax folding factor; Based on the reference radiance, the corrected rotation matrix, the fragment normal vector, the illumination vector, the luminance gain, and the overall reflectance coefficient, the formula for calculating the model radiance is as follows: ; in, Indicates time pixels The radiance of the model at that location, Indicates time The reference radiance, Represents pixels Corresponding fragment index fragment normal vector, Represents pixels The corresponding fragment index, Indicates time The illumination vector, Indicates brightness gain. Indicates fragment index At any moment The overall reflectance coefficient; Based on the 3D model, the corrected rotation matrix, the corrected translation vector, the occlusion depth, and the model radiance, the calculation formula for the virtual image is as follows: ; in, Indicates time pixels Virtual images at the location, Indicates the rendering operator. Representing a 3D model, Represents the occlusion depth matrix. This represents the radiance matrix of the model.

[0045] It should be noted that the depth modulation coefficient is a parameter that controls the intensity of the impact of reflection risk on occlusion depth. It is a user-defined parameter, with a preferred value of 0.5 meters and a range of 0.2 to 1 meter. This value can be adjusted based on the depth difference between the virtual model and the real scene; the greater the depth difference, the larger the value. The reference radiance is a radiometrically calibrated reference value for real-world lighting, which can be obtained in real-time by light sensors deployed within the scenic area. These sensors are typically deployed in open areas of the scenic area to avoid obstruction, and the preferred sampling frequency is 1 Hz to reflect changes in light intensity in real time. The brightness gain is a parameter that controls the degree of influence of wetting reflections on the brightness of the virtual model. It is a user-defined parameter, with a preferred value of 0.3 and a range of 0.1 to 0.5. This value can be adjusted based on the sunlight intensity at different times of day; the higher the light intensity, the larger the value. The rendering operator is an algorithm module that generates a virtual image based on the model, pose, depth, and brightness. It can be implemented using a conventional 3D rendering engine. The corrected depth value is the depth calculated based on the corrected pose and is used to generate pixel-level occlusion depth.

[0046] Specifically, the rendering operator uses a forward rendering pipeline based on a depth buffer. The depth test rule is set to pass if the depth value is less than or equal to the depth buffer value of the current pixel; that is, when the depth value of a fragment is less than or equal to the depth buffer value of the current pixel, the color and depth information of that pixel are updated. The reference radiance needs to correspond to the camera's exposure parameters. By pre-collecting light sensor data and camera image brightness data under different exposure parameters, a mapping relationship between the two is established to ensure that the brightness of the virtual model is on the same scale as the brightness of the real scene.

[0047] Specifically, the depth modulation coefficient is determined based on the depth difference range between the virtual model and the real scene to ensure that depth modulation does not cause obvious visual breaks. For example, when the depth difference range between the virtual model and the real scene is 2 meters, the preferred depth modulation coefficient is 0.5 meters. At this value, the depth modulation amplitude is moderate, effectively reducing erroneous hard occlusion caused by wet film ghost images, while avoiding obvious visual breaks between the virtual model and the real scene. When the depth difference range between the virtual model and the real scene is 4 meters, the preferred depth modulation coefficient is 1 meter. At this value, the depth modulation amplitude is larger, better adapting to larger depth differences. In practical applications, the depth modulation coefficient value can be adjusted through field testing to obtain the best visual effect.

[0048] Specifically, the brightness gain is dynamically adjusted based on the sunlight intensity at different times of day to ensure consistency between the virtual model and the real environment's lighting. For example, when the sunlight intensity is high at noon, a brightness gain of 0.3 is preferred, allowing the virtual model to exhibit a highlight effect similar to the real scene in wet, bright areas. When the sunlight intensity is low in the early morning or evening, a brightness gain of 0.2 is preferred, preventing the virtual model from being too bright and inconsistent with the real environment. In practical applications, a mapping table between light intensity and brightness gain can be established, automatically adjusting the brightness gain value based on real-time collected light intensity.

[0049] In one embodiment of the present invention, the specific calculation process of step S7 includes: The formula for calculating the camera center coordinates based on the corrected rotation matrix and the corrected translation vector is as follows: ; in, Indicates time The camera center coordinates Indicates time The corrected rotation matrix, Indicates time The corrected translation vector; The formulas for calculating the interaction ray vector and interaction ray coordinates based on the interaction pixels, the correction rotation matrix, and the intrinsic parameter matrix are as follows:

[0050] ;in, Indicates time Interactive ray vectors, The inverse matrix of the intrinsic parameter matrix. Indicates the interaction pixel at time. The horizontal pixel coordinates, Indicates the interaction pixel at time. The vertical pixel coordinates, Indicates the interaction pixel at time. homogeneous pixel vectors Indicates time Interactive ray coordinates, Indicates the ray distance parameter; Based on the interactive ray coordinates and the 3D model, the formula for calculating the interactive 3D coordinate points is as follows: ; in, Indicates time Interactive 3D coordinate points, Indicates time Ray distance parameters, Representing a 3D model Model points in This represents the variable that corresponds to the minimum value of the objective function; The formula for calculating augmented reality images based on the current image, mask matrix, and virtual image is as follows: ; in, Indicates time pixels Augmented reality images at the location, Indicates time pixels The current image at that location, Indicates time pixels The mask matrix at that location, Indicates time pixels Virtual images at that location.

[0051] It should be noted that interactive pixels are the image pixel locations touched, gazed upon, or pointed to by a visitor, and can be obtained through various methods such as touchscreens, eye-tracking devices, or gesture recognition systems. Camera center coordinates are the position of the camera's optical center in the world coordinate system. The interactive ray vector is the unit direction vector originating from the camera center and passing through the interactive pixels. The interactive 3D coordinate point is the intersection of the interactive ray and the 3D model, corresponding to the spatial location actually pointed to by the visitor. The visibility mask is a pixel-level weight matrix that controls the fusion ratio of real images and virtual content, with values ​​ranging from 0 to 1. A value of 0 fully displays the real image, a value of 1 fully displays the virtual content, and values ​​between 0 and 1 proportionally fuse the real image and virtual content.

[0052] Specifically, the intersection of the interactive ray and the 3D model is achieved using the method of ray-triangle intersection. The intersection of the interactive ray with each triangle in the 3D model is detected sequentially, the position and distance of the intersection point are calculated, and the effective intersection point closest to the camera is taken as the interactive 3D coordinate point. The transition range of the visibility mask is set to 0.1 meters. When the depth difference between the virtual model and the real scene is less than 0.1 meters, the value of the visibility mask transitions linearly between 0 and 1, achieving a smooth blending of virtual and real content.

[0053] Specifically, interactive pixels can be acquired through various methods, such as touchscreens, eye-tracking devices, or gesture recognition systems. For example, when using mobile devices, the touch position of the visitor can be collected via the touchscreen as the interactive pixel; this method is simple and easy to implement. When using AR glasses, the gaze position of the visitor can be collected via eye-tracking devices as the interactive pixel; this method is more natural and requires no manual operation. When gesture interaction is required, the pointing position of the visitor's gesture can be collected via a gesture recognition system as the interactive pixel; this method enables richer interactive operations. In practical applications, the appropriate interaction method can be selected based on the type of device and the user's needs.

[0054] Specifically, the visibility mask is generated based on a comparison between the depth of the virtual model and the depth of the real scene, prioritizing content with a smaller depth value. For example, when the depth value of the virtual model is less than that of the real scene, it means the virtual model is in front of the real scene, the visibility mask value is 1, and the virtual content is displayed; when the depth value of the virtual model is greater than that of the real scene, it means the virtual model is behind the real scene, the visibility mask value is 0, and the real content is displayed; when the depth values ​​of the virtual model and the real scene are similar, the visibility mask value is between 0 and 1, proportionally blending the real image and virtual content to achieve a smooth transition effect. In practical applications, the transition range of the visibility mask can be adjusted to make the blending of virtual and real content more natural.

[0055] It should be noted that the above embodiment extends the registration compensation results to the interaction layer, ensuring that the interaction point is mapped to the real 3D site surface rather than the water film virtual image layer. This avoids interference from the water film virtual image on the visitor's interactive operation, improving the accuracy of the interaction. Traditional AR interaction methods typically calculate the interaction point directly based on the initial camera pose. In areas near the tide line, the water film virtual image can cause errors in camera pose, resulting in the interaction point being mapped to the water film virtual image layer, leading to issues such as missed selections or triggering incorrect components. However, by extending the registration compensation results to the interaction layer and using the corrected camera pose to calculate the interaction point, it ensures that the interaction point is accurately mapped to the real 3D site surface. This approach significantly improves the visitor's interactive experience and reduces operational errors.

[0056] In one embodiment of the present invention, such as Figure 2 As shown, an interactive display system for tourism projects based on 3D modeling and augmented reality includes: The fragment center projection module acquires the current image, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and 3D model. It then performs coordinate transformation by combining the intrinsic parameter matrix, initial rotation matrix, and initial translation vector to project the fragment center point of the 3D model onto the current image, thereby obtaining the initial pixel point. The comprehensive reflection coefficient calculation module calculates the comprehensive reflection coefficient based on the fragment elevation, fragment normal vector, tide height, line-of-sight vector, and illumination vector. The texture matching module matches the initial pixel points, the current image, and the model texture template to calculate the image gradient matrix and the corresponding pixel points of the fragments. The parallax folding coefficient calculation module calculates the parallax folding coefficient based on the fragment elevation, tide height, initial pixel, tide line tangential vector, image gradient matrix, and comprehensive reflection coefficient. The pose solving module calculates the pose solving weights based on the disparity folding coefficients, and combines the pose solving weights with the corresponding pixel points of the fragments to calculate the corrected rotation matrix and the corrected translation vector. The virtual image generation module calculates the occlusion depth and model radiance based on the corrected rotation matrix, corrected translation vector, disparity folding coefficient, and comprehensive reflectance coefficient to generate a virtual image. The augmented reality image generation module obtains interactive pixels and a mask matrix. Based on the interactive pixels, the corrected rotation matrix, the corrected translation vector, and the 3D model, it calculates the interactive 3D coordinates. Then, it combines the mask matrix with the current image and the virtual image to generate an augmented reality image.

[0057] It should be noted that the actual deployment and application of this invention is divided into two stages: preliminary preparation and system operation. In the preliminary preparation stage, firstly, a high-precision 3D model of the coastal tidal site is generated using laser scanning technology. This model is then fragmented according to a mesh triangular facet method, and the center point, normal vector, and texture template of each fragment are extracted. Next, tide level sensors and illumination sensors are deployed at suitable locations within the scenic area. The tide level sensors are deployed near the dock, and the illumination sensors are deployed in an open, unobstructed area, with the sampling frequency set to 1 Hz for both. Then, the camera intrinsic parameters are calibrated, and the reference reflectance coefficients of different stone materials at the site are measured to establish a mapping table between illumination intensity and brightness gain. Finally, the coordinates of multiple feature points within the scenic area are collected using RTK positioning equipment, establishing a correspondence between the world coordinate system and the geographical coordinates of the scenic area.

[0058] During system operation, tourists use mobile devices or lightweight AR glasses to open the AR navigation application. The system first obtains the initial camera pose through GPS positioning and pre-calibrated feature point information, and then collects camera images, tide data, and illumination data in real time. The system executes seven technical modules in sequence to synthesize the final AR display and output it.

[0059] The final output consists of two parts: First, a real-time AR display that accurately overlays 3D reconstructions of the ancient pier and wharf pillars onto the real scene without any floating or misalignment, and provides natural lighting effects in wet, high-light areas. Second, interactive 3D coordinate points. When a visitor touches a component of the ancient pier on the screen, the system accurately identifies the component and triggers the corresponding historical description and structural unfolding functions, preventing missed selections or triggering of incorrect components. For example, when a visitor clicks on a wharf pillar model 15 meters from the camera in a scene with a tide level of 1.2 meters, the system accurately calculates the 3D coordinates of the wharf pillar, displays textual descriptions of its construction date and historical role, and shows an animation of the wharf pillar's internal structure unfolding, which will not be elaborated upon here.

[0060] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain a formula that is closest to the real situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0061] The content of this embodiment has been described above, but this embodiment is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this embodiment, all of which are within the protection scope of this embodiment.

Claims

1. A method for interactive display of tourism projects based on 3D modeling and augmented reality, characterized in that, Includes the following steps: Step S1: Obtain the current image, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and 3D model. Combine the intrinsic parameter matrix, initial rotation matrix, and initial translation vector to perform coordinate transformation, project the fragment center point of the 3D model onto the current image, and obtain the initial pixel point. Step S2: Calculate the composite reflection coefficient based on fragment elevation, fragment normal vector, tide level, line-of-sight vector, and illumination vector. Step S3: Match the initial pixel points, the current image, and the model texture template to calculate the image gradient matrix and the corresponding pixel points of the fragments; Step S4: Calculate the parallax folding coefficient based on the fragment elevation, tide level, initial pixel, tide line tangential vector, image gradient matrix, and comprehensive reflection coefficient. Step S5: Calculate the pose solution weights based on the disparity folding coefficients, and combine the pose solution weights with the corresponding pixel points of the fragments to calculate the corrected rotation matrix and the corrected translation vector. Step S6: Calculate the occlusion depth and model radiance based on the corrected rotation matrix, corrected translation vector, disparity folding coefficient, and integrated reflection coefficient to generate a virtual image; Step S7: Obtain interactive pixels and mask matrix. Calculate interactive 3D coordinates based on interactive pixels, corrected rotation matrix, corrected translation vector, and 3D model. Combine the mask matrix with the current image and virtual image to generate an augmented reality image.

2. The interactive display method for tourism projects based on 3D modeling and augmented reality according to claim 1, characterized in that, Step S1 specifically includes the following steps: Step S101: Divide the 3D model into fragments and obtain the center point, normal vector and texture template of each fragment. Step S102: Multiply the fragment center point by the initial rotation matrix, and add the product to the initial translation vector to obtain the camera coordinate point; Step S103: Multiply the camera coordinates with the intrinsic parameter matrix and perform pinhole projection to obtain the initial pixel points; Step S104: Take the components of the camera coordinate point on the camera depth axis vector to obtain the initial depth value; Step S105: Write the initial pixel point and initial depth value into the slice data sequence.

3. The interactive display method for tourism projects based on 3D modeling and augmented reality according to claim 1, characterized in that, Step S2 specifically includes the following steps: Step S201: Multiply the fragment center point by the local vertical vector to obtain the fragment elevation; Step S202: Calculate the absolute value of the difference between the fragment elevation and the tide level, and perform an exponential calculation in combination with the tide line scale value to obtain the tide line wetness. Step S203: Combine the fragment center point with the initial rotation matrix and the initial translation vector to transform it into the camera coordinate system, and take the negative and normalize the transformation result to obtain the viewing vector. Combine the fragment normal vector and the lighting vector with the initial rotation matrix to transform them into the camera coordinate system to obtain the normal coordinate vector and the lighting coordinate vector. Step S204: Add the line-of-sight vector to the lighting coordinate vector, and normalize the result to obtain a half-angle vector; Step S205: Calculate the reflection angle coefficient based on the non-negative inner product of the normal coordinate vector and the line-of-sight vector, and multiply the tidal line wetting amount, the reflection angle coefficient, and the non-negative inner product of the normal coordinate vector and the half-angle vector to obtain the comprehensive reflection coefficient.

4. The interactive display method for tourism projects based on 3D modeling and augmented reality according to claim 1, characterized in that, Step S3 specifically includes the following steps: Step S301: Using the initial pixel as the value position, perform a smooth convolution on the current image to obtain a smooth image; Step S302: Calculate the spatial gradient of the smoothed image to obtain the image gradient matrix; Step S303: Move the model texture template within the candidate offset set and obtain the current image patch; Step S304: Calculate the sum of the mean-free product of the model texture template and the current image patch, the energy value of the model texture template, and the energy value of the current image patch; Step S305: Take the sum of the products with the mean removed as the numerator, and take the product of the energy value of the model texture template and the energy value of the current image block plus the stable value of the denominator as the denominator, and select the candidate offset when the ratio is the largest. Step S306: Add the initial pixel point to the candidate offset to obtain the pixel point corresponding to the fragment.

5. The interactive display method for tourism projects based on 3D modeling and augmented reality according to claim 1, characterized in that, Step S4 specifically includes the following steps: Step S401: Calculate the difference between the fragment elevation and the tide level, multiply the difference by the local vertical vector, and subtract the product from the fragment center point to generate the mirror coordinate point; Step S402: Project the mirror coordinates points together with the initial rotation matrix, initial translation vector and intrinsic parameter matrix to obtain mirror pixels, and calculate the pixel distance between the mirror pixels and the initial pixels. Step S403: Divide the pixel distance by the focal length scale to obtain the projection difference; Step S404: Generate a tidal line reference point along the tangential vector of the tidal line, project the tidal line reference point, and obtain the tidal line image vector by using the difference vector and the magnitude of the difference vector between the pixels of the tidal line reference point and the initial pixels. Step S405: Take the absolute value of the inner product of the image gradient matrix and the tide line image vector as the gradient numerator, and take the sum of the magnitude of the image gradient matrix and the steady value of the gradient as the gradient denominator. Step S406: Multiply the comprehensive reflection coefficient, the projection difference, and the ratio of the gradient numerator to the gradient denominator to obtain the parallax folding coefficient.

6. The interactive display method for tourism projects based on 3D modeling and augmented reality according to claim 1, characterized in that, Step S5 specifically includes the following steps: Step S501: Add the first factor to the disparity folding coefficient to obtain the weight denominator; Step S502: Divide one by the weight denominator to obtain the pose solution weights; Step S503: Project the fragment center point according to the rotation matrix and translation vector to be determined to obtain the projected pixel point. Step S504: Calculate the squared pixel error between the corresponding pixel of the fragment and the pixel to be projected; Step S505: Multiply the squared pixel error value with the pose solution weight to obtain the fragment weighted error, and sum the fragment weighted errors in the projectable fragment set to obtain the pose error sum. Step S506: Use the pose error and the rotation matrix and translation vector corresponding to the minimum value as the corrected rotation matrix and corrected translation vector.

7. The interactive display method for tourism projects based on 3D modeling and augmented reality according to claim 1, characterized in that, Step S6 specifically includes the following steps: Step S601: Multiply the fragment center point with the correction rotation matrix, add the product to the correction translation vector, and take the component of the sum on the camera depth axis vector to obtain the correction depth value. Step S602: Multiply the parallax folding coefficient by the depth modulation coefficient to obtain the depth increment; Step S603: Add the corrected depth value to the depth increment, and take the minimum value of the fragment set corresponding to the same pixel to obtain the occlusion depth. Step S604: Transform the fragment normal vector and the lighting vector respectively using the modified rotation matrix, and calculate the non-negative inner product of the two vectors after transformation; Step S605: Multiply the reference radiance, the non-negative inner product, and the sum of the products of the comprehensive reflectance coefficient and the radiance gain to obtain the model radiance; Step S606: Combine the 3D model, correct the rotation matrix, correct the translation vector, occlusion depth, and model radiance to render and generate a virtual image.

8. The interactive display method for tourism projects based on 3D modeling and augmented reality according to claim 1, characterized in that, Step S7 specifically includes the following steps: Step S701: Multiply the transpose of the corrected rotation matrix with the corrected translation vector, and take the negative of the product to obtain the camera center coordinates; Step S702: Expand the interactive pixel points into homogeneous pixel vectors, multiply the transpose of the modified rotation matrix, the inverse of the intrinsic parameter matrix, and the homogeneous pixel vectors in sequence, and normalize the product to obtain the interactive ray vector. Step S703: Combine the camera center coordinates, ray distance parameters, and interactive ray vectors to obtain the interactive ray coordinates; Step S704: Calculate the distance between the model point and the interactive ray coordinate in the 3D model, and take the model point and ray distance parameters corresponding to the minimum distance to obtain the interactive 3D coordinate point and ray distance parameters; Step S705: Multiply the current image by the result obtained by subtracting the mask matrix, multiply the virtual image by the mask matrix, and add the two products to generate an augmented reality image.

9. A tourism project interactive display system based on 3D modeling and augmented reality, characterized in that, Performing the interactive display method for tourism projects based on 3D modeling and augmented reality as described in any one of claims 1 to 8 includes: The fragment center projection module acquires the current image, intrinsic parameter matrix, initial rotation matrix, initial translation vector, and 3D model. It then performs coordinate transformation by combining the intrinsic parameter matrix, initial rotation matrix, and initial translation vector to project the fragment center point of the 3D model onto the current image, thereby obtaining the initial pixel point. The comprehensive reflection coefficient calculation module calculates the comprehensive reflection coefficient based on the fragment elevation, fragment normal vector, tide height, line-of-sight vector, and illumination vector. The texture matching module matches the initial pixel points, the current image, and the model texture template to calculate the image gradient matrix and the corresponding pixel points of the fragments. The parallax folding coefficient calculation module calculates the parallax folding coefficient based on the fragment elevation, tide height, initial pixel, tide line tangential vector, image gradient matrix, and comprehensive reflection coefficient. The pose solving module calculates the pose solving weights based on the disparity folding coefficients, and combines the pose solving weights with the corresponding pixel points of the fragments to calculate the corrected rotation matrix and the corrected translation vector. The virtual image generation module calculates the occlusion depth and model radiance based on the corrected rotation matrix, corrected translation vector, disparity folding coefficient, and comprehensive reflectance coefficient to generate a virtual image. The augmented reality image generation module obtains interactive pixels and a mask matrix. Based on the interactive pixels, the corrected rotation matrix, the corrected translation vector, and the 3D model, it calculates the interactive 3D coordinates. Then, it combines the mask matrix with the current image and the virtual image to generate an augmented reality image.

Images