Irregular mountainous multi-projection geometry photometric joint adaptive correction method and system

Abstract

The application discloses an irregular mountainous area multi-projection geometric photometric combined adaptive correction method and system, and belongs to the technical field of multi-projection display.The application firstly constructs a site digital twin model, extracts geometric parameters such as terrain projection distance and slope surface incidence angle;establishes a combined geometric-photometric transfer function, fuses a terrain attenuation factor to construct a light transmission matrix, and jointly solves geometric distortion and color compensation parameters by using convex optimization; makes spectral adaptive compensation on rock and vegetation partitions based on the twin model; and builds an incremental re-optimization closed loop of camera and sensor linkage, so that fast correction under the change of environment and equipment is realized.The application unifies modeling of terrain optical coupling relationship, eliminates color difference stripes in the splicing area, dynamically maintains picture effects without complete re-calibration, greatly improves visual consistency of the multi-projection picture in the irregular mountainous area, and reduces long-term operation and maintenance cost of real scene projects.

Description

Technical Field

[0001] This invention relates to the field of multi-projection display technology, and in particular to a method and system for joint adaptive correction of multi-projection geometry and photometry in irregular mountainous terrain. Background Technology

[0002] Real-life mountain-themed cultural tourism performances and outdoor mountain simulations often employ multiple high-lumen engineering projectors to construct giant spliced ​​images on irregular curved surfaces such as undulating mountains and rock walls. The varying elevation differences in mountains, along with the mixed materials of rocks, vegetation, and bare soil on the surface, can simultaneously cause problems such as geometric distortion of the projection, color difference between multiple projectors, and gaps between light and dark areas in the splicing. Most mainstream correction algorithms on the market are designed for flat indoor screens and struggle to adapt to the unique geometric and optical coupling relationships of mountainous terrain, while also lacking long-term adaptive adjustment capabilities for outdoor scenes.

[0003] Currently, mainstream multi-projection calibration primarily employs a step-by-step approach, combining geometric correction and color compensation. Commercial calibration equipment utilizes industrial cameras to perform image alignment and color equalization in stages; academic methods often employ FFD free deformation, binocular 3D reconstruction, and neural networks to optimize deformation accuracy and color matching effects, respectively. Some cutting-edge solutions introduce digital twins to complete 3D site modeling, combining this with camera acquisition loops to achieve fine-tuning of the basic image. Overseas technologies rely on embedded cameras to simplify calibration operations and adapt to small and medium-sized irregular curved surface projections. This type of technology relies on mature hardware such as 3D laser scanning, GPU real-time rendering, and spectral acquisition, and has proven successful in indoor exhibition halls and small building facade projections, enabling static basic splicing calibration.

[0004] Current technologies still have many shortcomings: 1. The step-by-step geometric and photometric calculations ignore the physical coupling effect of mountains. Terrain undulations simultaneously cause distance light attenuation and slope reflection differences, leading to mutual interference between the two optimization parameters and the appearance of brightness stripes and visible color differences in overlapping areas. 2. Color correction assumes uniform reflection on the projected surface and fails to distinguish the spectral reflectance characteristics of rocks and vegetation, resulting in obvious color bands at material boundaries and a significant decrease in the accuracy of correction in mountain scenes. 3. There is a lack of a long-term adaptive operation and maintenance mechanism. After changes in ambient light and equipment light attenuation, a complete set of color charts needs to be re-collected and calibrated, which is time-consuming and requires interrupting the performance. Although existing solutions can introduce digital twins to complete terrain modeling, they do not rely on the twin model for material zoning compensation, nor do they establish an incremental rapid re-optimization closed loop. When applied to large-scale mountain scenes, there are many implementation constraints and high operation and maintenance costs.

[0005] The industry urgently needs an integrated correction technology that can simultaneously handle deformation and color difference, adapt to diverse mountain surface materials, and be dynamically maintained over the long term, to ensure visual consistency of the giant screen image on the mountain. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a method and system for joint adaptive correction of geometric and photometric properties in irregular mountainous terrain. By constructing a joint geometric-photometric transfer function that integrates terrain parameters, geometric and color compensation parameters are solved simultaneously within a convex optimization framework. Surface material zoning compensation is achieved using digital twins, and a sensor-driven incremental re-optimization closed loop is established. This solves problems such as conflicting step-by-step correction parameters, color differences in mountainous materials, and cumbersome outdoor operation and maintenance calibration. It offers advantages such as high image consistency, real-time adaptive adjustment, and low long-term operation and maintenance costs.

[0007] This invention is implemented as follows:

[0008] It includes two aspects, the first aspect:

[0009] A joint adaptive correction method for multi-projection geometric photometry in irregular mountainous terrain includes the following:

[0010] Step S1, Data Acquisition and Preprocessing: Collect original point cloud data of the mountain, build a global coordinate system and complete multiple sets of point cloud registration preprocessing, and output a standard point cloud dataset under a unified coordinate system.

[0011] Step S2: Construct a digital twin model of the mountain projection area and complete material partitioning: Construct a three-dimensional mesh model of the mountain slope based on the standard point cloud dataset, solve the spatial parameters of non-node positions inside the mesh unit by bilinear interpolation, and solve the normal vector of the mountain slope in a unified manner; combine the real-world landform features and texture information of the mountain to complete the material region partitioning on the surface of the digital twin model and construct a mountain surface spectral reflectance database;

[0012] Step S3: Establish the geometric-photometric coupling transfer model and the multi-projection illumination transfer matrix: Based on the three-dimensional coordinates and unit normal vector of the slope obtained from the digital twin model, and combined with the coordinates of the fixed installation points of each projector, solve the geometric optical influence factor of the mountain slope pixel by pixel; adopt the third-order Bernstein basis FFD free deformation model, and solve the inherent color transfer function of the projector through the multi-slope differential calibration mechanism; based on the geometric optical influence factor, surface spectral reflectance, and Gaussian point spread function, construct the multi-projection unified illumination transfer matrix.

[0013] Step S4, convex optimization to jointly solve the geometric photometric compensation parameters: Based on the target display screen and the illumination transmission matrix, pixel smoothing constraints and regularization constraints are introduced to construct a unified convex optimization objective function and solve the geometric deformation compensation parameters and color photometric compensation parameters.

[0014] Step S5, Real-time Joint Correction Rendering Output: The obtained geometric deformation compensation parameters and color photometric compensation parameters are encapsulated into the graphics processor pixel shader, and the inverse mapping relationship from the original image pixels to the mountain projection slope pixels is established. The real-time input image is corrected and rendered frame by frame and then output.

[0015] Step S6, Sensor closed-loop incremental adaptive re-optimization: Real-time acquisition of on-site working condition data. When the ambient light brightness, color temperature or projector light output status meets the preset trigger conditions, the incremental re-optimization process is automatically triggered, the compensation parameters are updated and synchronized to the graphics processor shader in real time.

[0016] Furthermore, the acquisition of raw point cloud data of the mountain mentioned in step S1 includes:

[0017] A ground-based 3D laser scanner was used to scan the mountain slope from multiple stations to obtain high-precision topographic point clouds at close range.

[0018] Aerial oblique photogrammetry using a drone equipped with an RTK positioning module was employed to acquire images of a wide range of mountain top surfaces and distant contours and generate airborne point clouds.

[0019] No fewer than five ground feature control points with precise RTK coordinates are deployed on the ground in the projection area as coordinate system binding references; using the RTK ground feature control points as coordinate system binding references, coordinate unification and fusion of ground laser point cloud and UAV airborne point cloud are achieved.

[0020] Furthermore, the method for constructing the mountain surface spectral reflectance database in step S2 is as follows:

[0021] By combining the real-world landform features and texture information of the mountain, material areas are divided on the surface of the digital twin model to distinguish different surface material areas such as rocks, bare soil, vegetation, and artificial structures.

[0022] For each type of material region, at least 20 uniformly distributed sampling points are randomly selected, and RGB three-channel spectral reflectance data are obtained using a spectral acquisition device. The average value of the region is then used as the reference spectral reflectance for that type of material.

[0023] For reflectance parameters at arbitrary coordinate positions of the model, a bicubic spline interpolation method is used to achieve continuous query. The spline interpolation boundary adopts natural boundary conditions, where the second derivative is zero, to avoid abnormal oscillations in the interpolation results at the material partition boundary, and to construct a comprehensive mountain surface spectral reflectance database.

[0024] Furthermore, the step S3, which involves solving the projector's inherent color transfer function using a multi-slope differential calibration mechanism, specifically includes:

[0025] Select at least three hillside surfaces of different materials and background colors as calibration areas, and query the reflectance parameters of the corresponding locations on each calibration slope surface from the surface spectral reflectance database constructed in step S2. ;

[0026] Twenty-seven standard solid color test charts were projected sequentially onto each calibrated slope, and real-time images of each slope were simultaneously captured by an industrial camera. And capture the background image of the slope under a completely black projection screen. ;

[0027] The original image data collected from each slope were reverse-corrected, removing reflections from the mountain material, geometric light and shadow attenuation, and ambient background light interference pixel by pixel. The correction formula is as follows:

[0028] in, Geometric optics influence factor;

[0029] All slope-corrected sample data are merged into a global optimization sample set. A least-squares objective function with second-order difference smoothing regularization is constructed. The optimal control point parameters of the third-order Bernstein basis FFD model are solved to obtain the projector's inherent color transfer function. .

[0030] Furthermore, the multi-projection unified illumination transmission matrix mentioned in step S3 matrix elements Assign values ​​as follows:

[0031] in, Geometric optics influence factor, , The angle between the projected ray and the slope normal. This is the spatial distance from the optical center of the projector to the point on the slope. For the slope Surface spectral reflectance at each sampling point; It is a two-dimensional Gaussian point spread function. , For the first The projector pixel corresponds to the center of the light spot and the slope surface. Spatial Euclidean distance between sampling points This is the diffusion radius parameter.

[0032] Furthermore, regarding the satisfaction pixel pairs, let This enables the sparse construction of the illumination transmission matrix.

[0033] Furthermore, the aforementioned The truncation threshold is determined based on the following:

[0034] In the two-dimensional Gaussian point spread function, the energy at a distance of 3σ from the center of the light spot has decayed to 0.1% of the central peak, meaning that the vast majority of the energy of the entire light spot is concentrated within a radius of 3σ. After truncating the long-distance contribution with 3σ as the boundary, the storage complexity of the illumination transfer matrix decreases from... Down to , where K is the number of effective influencing pixels of a single projector pixel within the camera's field of view, and K is less than 10% of Nc, corresponding to a modeling accuracy loss of less than 1.1%.

[0035] Furthermore, the convex optimization objective function described in step S4 is:

[0036] in, This is the optimal projection compensation parameter matrix; Display the screen for the target; For illumination transfer matrix; It is the set of all adjacent pixel pairs; Adjacent pixels and The compensation parameter vector; Smoothing weights for adjacent pixels; is the Tikhonov regularization coefficient.

[0037] Furthermore, the preset triggering conditions in step S6 are: ambient light brightness fluctuation exceeds the reference value by ±15%, color temperature shift exceeds ±500K, or projector light output attenuation is greater than 5%; in the incremental re-optimization process, one set of full white uniform field pattern and one set of grayscale gradient pattern are projected, the local correction amount is solved based on the historical reference illumination transmission matrix and the current collected sample, and the update is completed by gradient descent with 50 iterations.

[0038] The second aspect:

[0039] An adaptive correction system for multi-projection geometric photometry of irregular mountain terrain includes a multi-source data acquisition and point cloud preprocessing module, which is used to acquire raw point cloud data of the mountain and perform preprocessing and coordinate registration and fusion, and output standard mountain elevation point cloud data in a unified coordinate system.

[0040] The digital twin modeling and material partitioning module is used to construct a three-dimensional mesh model of the mountain slope based on the point cloud data, solve the spatial parameters of non-node positions inside the mesh unit by bilinear interpolation, and construct a database of spectral reflectance of the mountain surface.

[0041] The geometry-photometric coupling transfer model construction module is used to solve the three-dimensional coordinates and unit normal vector of the slope based on the digital twin model, calculate the geometric optical influence factor for each projected pixel, solve the inherent color transfer function of the projector through the multi-slope differential calibration mechanism, and construct a unified illumination transfer matrix for multiple projections based on the geometric optical influence factor, surface spectral reflectance and Gaussian point spread function.

[0042] The convex optimization joint solution module is used to construct a unified convex optimization objective function based on the target display screen and the illumination transmission matrix, and simultaneously solve for the geometric deformation compensation parameters and color photometric compensation parameters.

[0043] The real-time correction rendering output module is used to encapsulate the compensation parameters into the graphics processor pixel shader, establish the inverse mapping relationship between the original image pixels and the mountain projection slope pixels, perform frame-by-frame correction rendering on the real-time input screen and output it to each engineering projector.

[0044] The sensor closed-loop incremental adaptive re-optimization module is used to collect on-site working condition data in real time. When the ambient light brightness, color temperature or projector light output status meets the preset trigger conditions, the incremental re-optimization process is automatically triggered to update the compensation parameters and synchronize them to the graphics processor shader in real time.

[0045] The beneficial effects of this invention are:

[0046] (i) Integrated geometry-photometry coupled modeling to eliminate parameter conflicts in step-by-step solution

[0047] This invention integrates geometric attenuation factor, surface spectral reflectance, and Gaussian point spread function into the multi-projection illumination transfer matrix, achieving deep fusion of geometric deformation and color-photometric characteristics at the physical modeling level. Traditional correction schemes solve geometric and photometric corrections independently and step-by-step, which easily leads to mutual interference and inverse cancellation between the two types of parameters, resulting in brightness stripes and color differences in the overlapping projection areas. This invention uses a unified convex optimization framework to achieve joint and synchronous solving of geometric and photometric parameters, fundamentally eliminating parameter coupling conflicts from step-by-step solutions, completely eliminating brightness banding and color shifts in multi-camera spliced ​​images, and significantly improving the consistency of the entire image.

[0048] (ii) Multi-slope differential calibration to eliminate interference from reflections of mountain materials

[0049] To address the shortcomings of traditional projection calibration methods, which assume uniform slope reflection characteristics and are unsuitable for complex mountainous terrain, this invention employs a multi-slope differential calibration mechanism. By selecting multiple mountain slopes with different materials and background colors for differentiated sampling, and combining this with a reverse calibration algorithm to remove material reflection differences between rocks, bare soil, and vegetation pixel by pixel, the calibration results retain only the projector's inherent color distortion, eliminating optical interference from the mountainous terrain. This effectively solves the problems of color banding and insufficient calibration accuracy at material boundaries in traditional solutions, significantly improving color reproduction accuracy in complex mountainous scenes.

[0050] (III) Sensor closed-loop incremental adaptive re-optimization to achieve long-term maintenance without downtime.

[0051] This invention establishes a closed-loop monitoring mechanism for environmental and equipment status. Relying on multiple sensors to collect real-time operating data, it automatically triggers an incremental re-optimization process when ambient light fluctuates, color temperature shifts, or equipment light decay exceeds a threshold. Compared to traditional solutions that require system shutdown, full re-sampling of color charts, and time-consuming calibration methods, this invention projects only a small number of simple test patterns, iteratively corrects based on historical benchmark parameters, and lightweightly completes parameter hot updates without interrupting the performance. This effectively solves the image quality drift problem in dynamic outdoor environments, achieving long-term stable system operation and significantly reducing manual maintenance costs and downtime losses.

[0052] In summary, this invention relies on high-precision digital twins of mountains and material partitioning optical compensation to construct a complete technical closed loop of calibration, coupling correction, and adaptive updating. It is suitable for large-size, irregular outdoor mountain projection scenarios, and its engineering practicality and scene adaptability are significantly better than traditional technical solutions.

[0053] The present invention will be explained in detail below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0054] Figure 1 This is the overall flowchart of the multi-projection geometric photometric joint adaptive correction method for irregular mountainous terrain according to the present invention;

[0055] Figure 2 This is a detailed flowchart of the calibration steps for the color transfer function of a third-order FFD projector in step S3 of embodiment 1 of the present invention;

[0056] Figure 3 This is a schematic diagram of the element calculation and 3σ truncation of the illumination transmission matrix T in step S3 of embodiment 1 of the present invention. Detailed Implementation

[0057] Example 1:

[0058] This embodiment presents a method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain, such as... Figure 1 As shown, it includes the following steps:

[0059] Step S1, Data Acquisition and Preprocessing:

[0060] Collect original point cloud data of the mountain, build a global coordinate system and complete multiple sets of point cloud registration preprocessing, and output a standard point cloud dataset under a unified coordinate system to unify the spatial reference of multiple devices, eliminate positional deviations of multi-source surveying data, and avoid spatial misalignment in subsequent modeling and projection calculations.

[0061] 1. Multi-source data acquisition

[0062] A terrestrial 3D laser scanner was used to scan the mountain slope from multiple stations, acquiring high-precision close-range topographic point clouds. This was combined with UAV oblique photogrammetry using an RTK positioning module to obtain images of the mountain top and distant contours, generating airborne point clouds. At least five ground feature control points with precise RTK coordinates were deployed in the projection area as coordinate system binding references. All engineering projectors and image acquisition industrial cameras had their installation locations pre-recorded, and coordinate input interfaces were reserved.

[0063] 2. Point cloud preprocessing

[0064] Statistical filtering was used to remove floating noise and invalid measurement points at a distance generated by scanning. The number of neighboring points for statistical filtering was set to 50, and the standard deviation threshold was set to 2.0. Voxel downsampling was used to uniformly thin the point cloud to reduce the amount of computation. The optimal value for the voxel grid side length was 0.05m-0.1m, which was determined based on the original point cloud density and scene size. By combining pass-through filtering and interactive selection, points that are not part of the main structure of the mountain, such as tree crown free points and temporary debris on site, were removed.

[0065] 3. Coordinate registration and point cloud fusion

[0066] Using the on-site RTK marker control points as a reference, all scanned point clouds, projector installation positions, and camera positions are converted to the same local engineering coordinate system to determine a unique coordinate origin and three-axis directions.

[0067] The SAC-IA algorithm is used to perform preliminary matching of multi-site cloud, airborne and ground point cloud, and the random sampling consensus algorithm is used to remove erroneous matching points to complete coarse alignment.

[0068] The GICP generalized iterative nearest point algorithm is used for fine-grained registration iteration. The operation ends when the registration error changes by less than 5 mm between two consecutive iterations, and the unified coordinate transformation parameters are obtained.

[0069] Based on the transformation parameters, all point clouds are stitched together and merged to generate a single complete, coordinate-unified, and pre-processed mountain facade point cloud dataset, which serves as the input source for the next step of digital twin modeling.

[0070] Step S2: Construct a digital twin model of the mountain projection area and complete material partitioning:

[0071] 1. Reconstruction of digital twin mesh model

[0072] Based on a standardized mountain point cloud dataset with preprocessed registration, a 3D mesh reconstruction of the mountain slope is achieved using a regular grid or quadrilateral grid. This enables precise digital replication of irregular mountain slopes, generating a continuous, void-free digital twin model that closely matches the real mountain slope, suitable for large-scale, irregular mountain-themed live-action performance projection scenes. Each node in the mesh model corresponds to the real spatial coordinates of the mountain, recording 3D spatial coordinates, slope texture information, and mesh normal vectors. For the spatial parameters of non-node locations within the mesh cells, accurate solutions are obtained using bilinear interpolation.

[0073] The mesh model uses mesh nodes as basic spatial sampling points. Each mesh node is bound to the real geospatial information of the mountain, including 3D spatial coordinates, slope texture and color information, and mesh patch normal vectors. Since the mesh nodes are discrete sampling points, there are no directly sampled parameters for any continuous slope position within the mesh unit. To achieve continuous solution of spatial parameters across the entire slope and ensure the accuracy of point-to-point correspondence of projected pixels, the spatial coordinates of any position within the mesh unit are solved using a bilinear interpolation method. The calculation formula is as follows:

[0074]

[0075] In the formula: The three-dimensional spatial coordinates of the mountain at any interpolation point; The spatial coordinates of the four corners of the grid cell; These are the normalized relative offset coefficients of the interpolation points within the grid cell in the horizontal and vertical directions, respectively. The closer the offset coefficient is to 0, the closer it is to the starting corner of the grid cell; the closer it is to 1, the closer it is to the ending corner of the grid cell.

[0076] After solving for the grid spatial coordinates, the slope normal vector is uniformly solved for subsequent calculations of the ray incident angle and geometric optical attenuation factor. The grid slope normal vector is the average of the normal vectors of two adjacent grid patches, and is then normalized to obtain the accurate slope normal vector parameters.

[0077] 2. Surface material zoning and spectral reflectance library construction

[0078] Combining the real-world terrain features and texture information, material regions were divided on the surface of the digital twin model, distinguishing different surface materials such as rock, bare soil, vegetation, and man-made structures. For each type of material region, at least 20 evenly distributed sampling points were randomly selected. If the area of ​​a certain material region is less than 1... The number of sampling points can be appropriately reduced, but not less than 10, to ensure statistical representativeness. RGB three-channel spectral reflectance data are acquired using a spectral acquisition device, and the regional mean is calculated as the baseline spectral reflectance for this type of material. For reflectance parameters at arbitrary coordinate positions in the model, a bicubic spline interpolation method is used to achieve continuous querying. The spline interpolation boundary adopts a natural boundary condition (second derivative is zero) to avoid abnormal oscillations in the interpolation results at material partition boundaries, thus constructing a comprehensive and high-precision mountain surface spectral reflectance database.

[0079] This step completes the accurate conversion of discrete point clouds of the mountain into a continuous three-dimensional curved surface model, and completes the spatial and optical parameters of the entire slope of the irregular mountain. It solves the shortcomings of traditional projection, which only uses discrete point correction and ignores the continuity of the mountain surface and the optical differences of materials. It provides complete, reproducible and high-precision digital model and optical data support for subsequent geometric-photometric coupling modeling, precise photometric compensation in different areas, and global pixel alignment correction.

[0080] Step S3: Establish the geometry-photometric coupling transfer model and the multi-projection illumination transfer matrix:

[0081] 1. Calculation of geometric influence factors of mountainous terrain

[0082] Based on the three-dimensional coordinates and unit normal vector of the slope obtained from the digital twin model, and combined with the coordinates of the fixed installation points of each projector calibrated on-site, the spatial geometric and optical influence parameters of the mountain slope are solved pixel by pixel, quantifying the influence of the mountain's spatial distance and slope orientation on the attenuation of projected light. Let the first... The fixed installation space coordinates of the projector are: The three-dimensional spatial coordinates of any projection point on the mountain slope are: The Euclidean distance from the optical center of the projector to the point of action on the mountain slope is calculated using the following formula:

[0083]

[0084] In the formula: For the first The straight-line distance from the projector to the projection point on the hillside; The three-dimensional coordinates of the projection points on the mountain slope; For the first The three-dimensional coordinates of the installation and positioning of the projector.

[0085] Based on the incident vector of the light ray and the normal vector of the slope, the cosine of the incident angle of the projected light ray is calculated to characterize the light reception efficiency of the slope. The calculation formula is as follows:

[0086]

[0087] In the formula: For the first The angle of incidence between the projector's beam and the mountain slope; Let be the incident ray vector from the projector pointing towards the projection point on the mountain; This represents the unit normal vector of the current position of the mountain slope; the formula as a whole characterizes the degree of fit between the incident direction of the light and the slope normal.

[0088] Combining the inverse square attenuation principle of light rays from distance with the Lambertian diffuse reflection imaging principle in optics, and integrating the dual influencing factors of distance attenuation and slope orientation, a unique geometric optical influence factor adapted to irregular mountains is constructed, while simultaneously shielding the ineffective areas obscured by the mountain. The calculation formula is as follows:

[0089]

[0090] In the formula: The geometrical optical influence factor is the factor at any location on the mountain slope; when the calculated value is... At that time, it was determined that the current slope location was backlit, obstructed by mountains, and had no effective projected light rays, so it was forced to... It is not involved in subsequent photometric compensation calculations.

[0091] 2. Projector color transfer function calibration

[0092] A third-order Bernstein-based FFD free-deformation model is used to characterize the inherent nonlinear distortion of the projector's RGB three-dimensional color space, eliminating terrain and environmental interference, and establishing the device's forward color transfer function. The third-order Bernstein single-channel model contains 4 control points, and the RGB three-channel global control points total [number missing]. To balance accuracy and computing power, a uniform selection was made. A total of 27 core nodes are used as optimization variables, and the remaining nodes are solved by trilinear interpolation. The global boundary control points adopt identity mapping constraints to suppress boundary color distortion.

[0093] A multi-slope differential calibration mechanism was used to isolate the background color of the mountain and the interference of light and shadow. The specific process is as follows:

[0094] 1) Select at least three typical slopes with different materials and background colors within the mountain projection area as the calibration area, and record the total number of calibration slopes as . , .definition To calibrate the slope serial number, the spectral reflectance of each area was measured in advance. The geometric optical factor at each pixel position is calculated using the following formula:

[0095]

[0096] In the formula: The geometrical optical influence factor is the factor at any location on the mountain slope. The angle between the incident light rays emitted from the projector and the local normal of the current slope surface; This is the Euclidean spatial distance from the optical center of the projector to the corresponding pixel on the slope.

[0097] 2) Project 27 standard pure color test cards onto each calibration slope in sequence, and record the true RGB values ​​of the 27 color cards. Known and uniformly distributed within the RGB color gamut, it can fully cover the color range of conventional performance projection; real-time imaging images of each slope are simultaneously acquired using high-precision industrial cameras to obtain raw imaging data. .

[0098] 3) Perform reverse correction on the raw image data collected from each slope, removing reflections from the mountain material, geometric light and shadow attenuation, and ambient background light interference pixel by pixel, and extracting pure color samples containing only projector distortion. Pre-collect background images of the slopes under a completely black projection screen. The correction formula used to subtract ambient light noise is as follows:

[0099]

[0100] In the formula: the random sampling noise of the image is suppressed by calculating the mean of the ROI region, and the pure device color response value corresponding to a single slope and a single color card is obtained. The corrected sample data corresponding to all slopes of different materials and all color cards are merged to construct a unified global optimized sample set to ensure the diversity and universality of the calibration data.

[0101] 4) Construct a least-squares objective function with second-order difference smoothing regularization, and solve for the optimal parameters of 27 core control points:

[0102]

[0103] in, To determine the total number of slopes, ; For the core control point parameter vector, ; For the first The RGB true value vector of a standard color chart; For the first The first slope Pure color response vector after calibration with a color chart; Forward mapping function of FFD model; Let be the Euclidean norm of the vector; A three-dimensional index for 27 core control points; It represents the second-order difference of the displacements of adjacent control points; The regularization coefficient is . .

[0104] Its unfolded form is as follows:

[0105] The initial value for the iteration is set to the unit mapping, and the Levenberg-Marquardt algorithm is used to solve it. The initial value of the damping factor in the algorithm is set to... The damping factor is adaptively adjusted based on the changes in the iterative residuals. When the residuals decrease, the damping factor is multiplied by 1 / 3; when the residuals increase, the damping factor is multiplied by 2. The convergence condition is that the relative change in the objective function is less than 1 / 3. Or iterate up to 200 times. Optimal parameters... Substituting into the model, we obtain the projector's inherent color transfer function. .

[0106] 3. Construction of Multi-Projection Illumination Transmission Matrix

[0107] This step establishes the mapping relationship between projector pixels and mountain slope imaging pixels, and constructs a unified illumination transmission matrix for multiple projections. .

[0108] In this section, the geometric attenuation factor The definitions used in the "Calculation of Geometric Influence Factors for Mountainous Areas" procedure are adopted uniformly:

[0109] The backlit or shaded areas ( ) is judged as To simplify notation, it is uniformly denoted as [insert notation here] in color transfer function calibration and illumination transfer matrix construction. or All three have the same physical meaning, and all include the dual effects of inverse square distance decay and slope orientation (including shading criterion).

[0110] Define the projector output pixel vector (Total effective pixels of a single projector) and camera imaging pixel vector (Total number of effective pixels in the image captured by the camera). The illumination transmission matrix In the middle, located in the first Line 1 Column elements Characterizing the first The projector pixel pair The brightness contribution value of each camera pixel.

[0111] Matrix elements Assign values ​​as follows:

[0112]

[0113] in:

[0114] (1) For the first The projector pixel to the slope The geometric attenuation factor of each sampling point is calculated according to a unified formula. Calculation, where This is the spatial distance from the optical center of the projector to the point on the slope. The angle between the projected ray and the slope normal, and the shaded or obscured area ( Forced reset to zero;

[0115] (2) For the slope The surface spectral reflectance of each sampling point is taken from the surface spectral reflectance database constructed in step S2;

[0116] (3) It is a two-dimensional Gaussian point spread function. ,in For the first The projector pixel corresponds to the center of the light spot and the slope surface. Spatial Euclidean distance between sampling points The diffusion radius parameter is 0.5 to 1.5 times the physical size of a single pixel of the projector on the slope. The physical size of a single pixel of the projector on the slope is determined by the projector resolution, projection ratio and spatial distance from the projector to the slope, or by projecting a standard checkerboard calibration board and measuring the actual size of the slope corresponding to a single grid point.

[0117] To satisfy pixel pairs, let This achieves the sparse construction of the illumination transmission matrix. The selection of the cutoff threshold is based on the following: In the two-dimensional Gaussian point spread function, the distance from the center of the light spot... The energy at that point has decayed to about 0.1% of the central peak, meaning that over 99% of the energy in the entire light spot is concentrated in... Within the radius. Therefore, with By truncating long-distance contributions to the bounds, the storage complexity of the illumination transfer matrix can be reduced from... Down to ,in This refers to the number of effective influencing pixels of a single projector pixel within the camera's field of view, and The corresponding modeling accuracy loss is less than 1.1%, which significantly reduces matrix storage and iterative computation overhead while ensuring correction accuracy.

[0118] Step S4, convex optimization to jointly solve for the geometric photometric compensation parameters:

[0119] 1. Construct a joint optimization objective function

[0120] By combining the target display image and the illumination transfer matrix, pixel smoothing constraints and regularization constraints are introduced to construct a unified convex optimization objective function that takes into account image consistency, smoothness, and solution stability. This enables the simultaneous solution of geometric deformation and color photometric compensation parameters. The objective function formula is as follows:

[0121]

[0122] In the formula: This is the optimal projection compensation parameter matrix; Display the screen for the target; It is the set of all adjacent pixel pairs; Adjacent pixels and compensation parameter vector This is the geometry-photometric coupled illumination transfer matrix; Smoothing weights for adjacent pixels, The preferred value range is , These are the Tikhonov regularization coefficients, used to prevent overfitting in the solution; the preferred range of values ​​is [value range missing]. Constraints Match the projector's normal brightness output range to avoid overexposed or underexposed images.

[0123] 2. Gradient descent iterative solution

[0124] The projected gradient descent algorithm is used for iterative solution. The iteration step size is adaptively determined by Armijo line search to ensure iterative convergence stability. The iterative update formula is as follows:

[0125]

[0126] In the formula: For the first The compensation parameter matrix for the next iteration; This is the updated parameter matrix; is the adaptive iteration step size; clamp is the interval truncation function, which forces the constraint parameters to be within the effective output interval.

[0127] Set an iterative convergence criterion and a maximum number of iterations. The convergence criterion is set when the error between two consecutive iterations is less than... The calculation may terminate when the number of iterations reaches 500, and the geometric deformation compensation parameters and color photometric compensation parameters will be output synchronously in the end.

[0128] This step achieves joint and synchronous solution of geometric and photometric parameters through an integrated convex optimization framework, eliminating the problems of parameter cancellation and insufficient correction accuracy caused by traditional step-by-step iteration, and greatly improving the overall consistency of the mountain projection image. At the same time, regular constraints ensure stable solution results and natural image transitions, adapting to the needs of large-scale mountain real-scene cinematic image quality.

[0129] Step S5, GPU real-time joint correction rendering output:

[0130] 1. Correction parameter encapsulation mapping

[0131] The geometric deformation field and color compensation parameters obtained in step S4 are uniformly encapsulated into the GPU pixel shader, and a reverse mapping relationship is established from the original image pixels to the mountain projection slope pixels to achieve global parameter reuse.

[0132] The reverse mapping relationship is constructed from the illumination transmission matrix in step S3. The Moore-Penrose pseudoinverse or regularized inverse matrix is ​​given.

[0133] Given the aforementioned light transmission matrix Usually non-square matrix ( Furthermore, there may be ill-conditioned condition numbers. This invention uses the Tikhonov regularization method to construct its regularized inverse matrix, which is used to inversely solve the projector input compensation signal from the target image. The calculation formula is as follows:

[0134]

[0135] in, This is the regularized inverse of the illumination transfer matrix. The illumination transmission matrix constructed in step S3, The total number of pixels in the camera image. This represents the total number of effective pixels in a single projector. for The transpose of the matrix; It is the identity matrix; The regularization coefficient controls the singularity suppression strength during matrix inversion, and its preferred value range is [value range missing]. The specific value is based on The condition number is dynamically adjusted; the larger the condition number, the better. The value increases accordingly.

[0136] 2. Real-time frame-by-frame correction calculation

[0137] For the real-time input of the original projected image, geometric distortion correction is first performed by pre-calculating the deformation field. Then, combined with the reflectivity of the mountain surface and the geometric optical influence factor, the precise color compensation parameters are solved in reverse using the Newton-Raphson iteration method. The number of iterations does not exceed 10 to ensure computational efficiency. The image correction calculation formula is as follows:

[0138]

[0139] In the formula: To finally correct the color parameters of the output pixels; The inverse function for projector color transfer; Target standard pixel color; This represents the surface spectral reflectance at the current location of the slope. This represents the geometrical optical influence factor at the corresponding location.

[0140] 3. Real-time output of high-definition images

[0141] The corrected image signal is synchronously output to each engineering projector, and the real-time rendering of the correction algorithm is realized by relying on the parallel computing power of the GPU to meet the requirements of continuous and smooth playback of live performances.

[0142] This step leverages GPU parallel computing power to achieve lightweight real-time computation of the correction algorithm, combining offline optimization parameters with online real-time rendering depth. While ensuring high-precision correction effects for multiple mountain projections, it solves the problems of insufficient computing power and screen stuttering / latency in large-size real-scene projections, and stably outputs high-quality, cinematic-grade mountain projection images.

[0143] Step S6, Sensor closed-loop incremental adaptive re-optimization

[0144] 1. Adaptive update triggering mechanism

[0145] The system collects real-time on-site operating data through ambient light sensors, color temperature sensors, and equipment status monitoring modules. When the ambient light brightness fluctuation exceeds the baseline value by ±15%, the color temperature deviation exceeds ±500K, or the projector light output attenuation is greater than 5%, the incremental re-optimization process is automatically triggered.

[0146] 2. Lightweight incremental correction update

[0147] No full recalibration is required; only two simple test patterns are projected to complete local image acquisition. These simple test patterns are: a full-white uniform field pattern (for detecting brightness decay) and a grayscale gradient pattern (for detecting color temperature shift). The projection and acquisition of both patterns are completed within one second; based on the historical reference illumination transfer matrix. Compared with the currently collected samples Solve for the local correction amount ,satisfy To minimize this, a lightweight gradient descent approach with only 50 iterations is used to update the global optical transmission parameters. Simultaneously, the optimal compensation parameters from the previous round are used as initial values ​​to reduce computational complexity. The maximum number of iterations is set to 50, enabling rapid parameter convergence and updates.

[0148] 3. Parameters are updated in real time.

[0149] The updated compensation parameters are synchronized to the GPU shader in real time, without any downtime or interruption of the performance projection playback, achieving seamless adaptive correction in the background.

[0150] This step establishes an incremental closed-loop optimization mechanism that links the environment and equipment status, eliminating the drawbacks of traditional mountain projection which requires shutdown for full recalibration and is cumbersome to maintain. It can adaptively adapt to the dynamic changes in outdoor mountain light and shadow and equipment aging, ensuring the consistency and high quality of real-scene projection images in the long term, and significantly reducing the operation and maintenance costs of outdoor performance projects.

[0151] Step S7, Verification of calibration and correction results

[0152] To verify the effectiveness of the calibration and correction method of this invention, a verification experiment was conducted in a real mountain projection scene:

[0153] (1) Color calibration accuracy verification: After completing the FFD calibration in step S3-2, project 12 additional verification color cards that were not included in the calibration, and calculate the CIEDE2000 color difference value between the output image after projection correction and the standard color card. Average color difference before calibration Average color difference after calibration The maximum color difference does not exceed 5.0, and the verification calibration method can effectively eliminate the inherent color deviation of the projector.

[0154] (2) Verification of consistency of multi-projector splicing: Select no less than 50 evenly distributed test points in the overlapping area of ​​adjacent projectors, measure the luminance and chromaticity values ​​of each test point after calibration, and calculate the root mean square error of luminance in the overlapping area. (Based on the target luminance normalized value), color difference The results show that the illumination transmission matrix and joint optimization method can effectively eliminate bright bands and color differences in multi-projection splicing.

[0155] (3) Real-time rendering performance verification: Run step S5 to correct the rendering process on the GPU parallel computing platform for no less than 60 minutes. Monitor the smoothness of the screen output, without stuttering or tearing, to verify that the present invention meets the performance requirements of real-time playback of live performance.

[0156] Example 2:

[0157] This embodiment presents a joint adaptive correction system for multi-projection geometric photometry in irregular mountainous terrain, based on the adaptive correction method described in Embodiment 1, as follows:

[0158] I. System Overall Architecture

[0159] The system adopts a modular layered architecture, which includes, from top to bottom: a data acquisition layer, a modeling layer, a model calculation layer, a real-time rendering layer, an adaptive operation and maintenance layer, and a central control and data management platform.

[0160] II. Composition and Functions of Each Module

[0161] Module 1: Multi-source data acquisition and point cloud preprocessing module

[0162] The module includes a 3D laser scanner, an UAV oblique photography system equipped with an RTK positioning module, no less than 5 ground feature control points with RTK coordinates, a high-precision industrial camera, and a point cloud preprocessing submodule (including statistical filtering, voxel downsampling, and pass-through filtering units), which is used to perform the multi-source data acquisition, point cloud preprocessing, and coordinate registration and fusion functions described in step S1 of Embodiment 1, and output a standard mountain facade point cloud dataset in a unified coordinate system.

[0163] Module 2: Digital Twin Modeling and Material Partitioning Module

[0164] The module includes a regular grid reconstruction submodule (which uses a regular grid or quadrilateral grid to reconstruct the three-dimensional grid of the mountain slope and achieves continuous solution of the spatial parameters inside the grid cell through bilinear interpolation), a normal vector calculation submodule, and a surface material zoning and spectral reflectance database construction submodule (which uses bicubic spline interpolation to achieve continuous reflectance query), used to execute the digital twin modeling and material zoning function described in step S2 of Example 1, and output a digital twin grid model and a full-coverage surface spectral reflectance database.

[0165] Module 3: Geometric-Photometric Coupling Transfer Model Construction Module

[0166] The module includes:

[0167] 3.1 Geometric Optical Influence Factor Calculation Unit: This unit performs the function described in step S3-1 of Example 1, calculating the geometric optical influence factor pixel by pixel based on the three-dimensional coordinates of the slope and the coordinates of the projector installation point obtained from the digital twin model. And shield areas where the mountain's obstruction is ineffective;

[0168] 3.2 Projector Color Transfer Function Calibration Unit: This unit performs the function described in step S3-2 of Example 1. It uses a third-order Bernstein-based FFD free-deformation model and a multi-slope differential calibration mechanism to solve for the optimal parameters of 27 core control points, thereby obtaining the projector's inherent color transfer function. ;

[0169] 3.3 Multi-projection illumination transmission matrix construction unit: used to perform the functions described in steps S3-3 of Embodiment 1, and construct a multi-projection unified illumination transmission matrix. Matrix elements are arranged in order to Assign values, and for those that satisfy Perform sparsification truncation on the pixel pairs.

[0170] Module 4: Convex Optimization Joint Solution Module

[0171] The module is used to perform the function described in step S4 of embodiment 1, and to construct a unified convex optimization objective function that takes into account image consistency, smoothness, and solution stability:

[0172]

[0173] The projected gradient descent algorithm combined with Armijo line search is used for iterative solution. The convergence condition is that the error between two consecutive iterations is less than 1. If the number of iterations reaches 500, the geometric deformation compensation parameters and color luminance compensation parameters will be output simultaneously.

[0174] Module 5: GPU Real-Time Correction Rendering Output Module

[0175] The module includes:

[0176] 5.1 Correction Parameter Encapsulation and Mapping Unit: This unit performs the function described in step S5 of Embodiment 1, encapsulating the compensation parameters into the GPU pixel shader to establish a reverse mapping relationship between the original image pixels and the projected hillside pixels; the reverse mapping relationship is determined by the illumination transfer matrix. Tikhonov regularized inverse matrix Give;

[0177] 5.2 Frame-by-Frame Real-Time Correction Calculation Unit: This unit performs the function described in step S5 of Example 1, solving for the color compensation parameters using Newton's iteration method. The number of iterations does not exceed 10. The correction formula is as follows: ;

[0178] 5.3 High-definition real-time output unit: used to synchronously output the corrected image signal to each engineering projector, with an overall rendering frame rate of no less than 60fps and a single frame delay of no more than 16.67ms.

[0179] Module 6: Sensor Closed-Loop Incremental Adaptive Re-optimization Module

[0180] The module includes:

[0181] 6.1 Adaptive update triggering unit: including ambient light sensor, color temperature sensor and device status monitoring module, used to perform the function described in step S6 of embodiment 1, automatically triggering incremental re-optimization when the ambient light brightness fluctuation exceeds the reference value ±15%, the color temperature offset exceeds ±500K or the projector light output attenuation is greater than 5%.

[0182] 6.2 Lightweight Incremental Correction Update Unit: Used to perform the function described in step S6 of Example 1, projecting two sets of test patterns: a full white uniform field and a grayscale gradient, based on the historical reference illumination transfer matrix. Compared with the currently collected samples Solving for local corrections The update is completed using 50 lightweight gradient descent iterations.

[0183] 6.3 Parameter Real-time Hot Update Unit: Used to synchronize the updated compensation parameters to the GPU shader in real time without any downtime.

[0184] Module 7: Central Control and Data Management Platform

[0185] The platform includes a global parameter configuration unit, a data storage and management unit, a status monitoring and visualization unit, and an optical fiber communication bus. It is bidirectionally connected to modules 1 to 6 via the communication bus to realize global parameter configuration, data storage, and status monitoring.

[0186] III. Connection Relationships Between Modules

[0187] 1. The output of module 1 is connected to the input of module 2 to output a standardized mountain elevation point cloud dataset;

[0188] 2. The output of module 2 is connected to the geometric factor calculation unit and color calibration unit of module 3 respectively, and outputs a digital twin mesh model and reflectivity database;

[0189] 3. The output of module 3 is connected to the input of module 4, outputting the illumination transmission matrix. and color transfer function ;

[0190] 4. The output of module 4 is connected to the input of module 5, outputting the optimal compensation parameter matrix. ;

[0191] 5. The output of module 5 is connected to each engineering projector to output the calibrated real-time image signal;

[0192] 6. The input of module 6 is connected to the ambient light sensor, color temperature sensor and equipment status monitoring module, and the output is connected to the calibration parameter encapsulation mapping unit of module 5;

[0193] 7. Module 7 is bidirectionally connected to modules 1 through 6 via a communication bus.

[0194] IV. System Workflow

[0195] The system's workflow includes an offline calibration phase and an online operation phase:

[0196] Offline calibration stage: collect and preprocess point cloud data through module 1 → build digital twin model and reflectivity database through module 2 → build geometry-photometric coupling transfer model through module 3 → perform convex optimization joint solution through module 4 → encapsulate compensation parameters into GPU shader through module 5.

[0197] During the online operation phase: Module 5 performs frame-by-frame correction and rendering of the real-time input screen → Module 6 monitors the environment and device status in real time, and automatically performs incremental re-optimization and hot updates compensation parameters in real time when the trigger conditions are met.

[0198] Example 3:

[0199] This embodiment uses an irregular mountain landscape performance project as the application scenario. The projection area of ​​the mountain is approximately 200m × 150m, with a maximum drop of about 30m. The surface is distributed with three typical materials: rock, vegetation, and bare soil. The projection system consists of eight 4K RGB pure laser engineering projectors, which are set up around the front of the mountain slope.

[0200] The specific implementation of the calibration process in this embodiment is as follows:

[0201] Three typical slopes of different materials within the mountain projection area were selected as calibration areas: Area A is a grayish-brown exposed rock slope, Area B is a dark green vegetation-covered slope, and Area C is a yellowish-brown bare soil slope. In the digital twin model constructed in step S2, the spectral reflectance parameters corresponding to the above three areas were queried and denoted as follows: , , .

[0202] Twenty-seven standard pure color test charts were projected sequentially onto three calibrated slopes. The color chart sequence started with pure black (RGB value (0,0,0)) and successively covered red, green, blue primary colors and their intermediate mixed colors, up to pure white (255,255,255), evenly distributed within the RGB color gamut. Each time a color chart was projected, an industrial camera was simultaneously triggered to capture an image of the corresponding slope, obtaining raw image data. .

[0203] Before the formal calibration and data acquisition, background images of each slope surface were pre-acquired under a completely black projection screen. Used to remove ambient light noise. Reverse correction is performed on the raw image data collected from each slope, removing reflectivity of the mountain material pixel by pixel. and geometric optical factors To eliminate interference, the pure color response values ​​corresponding to a single slope and a monochrome card were obtained. The 81 sets of corrected sample data from three slopes and 27 color cards were merged into a global optimized sample set.

[0204] The Levenberg-Marquardt algorithm is used to solve for the control point parameters of the FFD model. The initial values ​​for the iterations are set to unit mapping, and the initial value of the damping factor is set to 10⁻³. When the relative change of the objective function is less than 10⁻³, the algorithm is applied. 5 The iteration may terminate after 200 iterations. Substituting the optimal parameters into the FFD model yields the inherent color transfer function of the projector. .

[0205] In this embodiment, 81 samples from three slopes made of different materials are merged into a single global optimization sample set. The optimization objective is to minimize the joint residual of all samples, thus obtaining a unique set of globally optimal control point parameters. The sample data from the three slopes are mutually constrained and verified during the optimization process. The residual of any slope sample participates in the gradient calculation of the joint objective function. The final control point parameters minimize the total residual of all slopes, thereby avoiding the bias in the calibration results of a single-material slope on the inherent characteristics of the equipment.

[0206] Example 4:

[0207] This embodiment continues the application scenario of embodiment 3. The live performance lasts from evening to late at night, for about 90 minutes, and the system is in a continuous running state.

[0208] (a) Real-time monitoring by sensors

[0209] Ambient light sensors are deployed above the audience area and on both sides of the hillside, color temperature sensors are installed near the projectors, and equipment status monitoring modules are integrated inside each projector. These three types of sensors continuously collect on-site operating data at a sampling frequency of no less than 1Hz and transmit it to the central control platform in real time.

[0210] (ii) Trigger judgment and incremental update

[0211] Forty-five minutes into the performance, as darkness fell, the ambient light intensity decreased by approximately 18% compared to the start of the performance, and the color temperature shifted from 5500K to 4200K. The ambient light sensor detected that the brightness fluctuation exceeded the preset threshold (reference value ±15%), and the color temperature sensor detected that the color temperature shift exceeded the preset threshold (±500K). The central control platform automatically triggered an incremental re-optimization process.

[0212] The re-optimized process automatically performs the following operations: sequentially projecting one set of pure white uniform field patterns and one set of grayscale gradient patterns, while simultaneously acquiring slope images using an industrial camera. This is based on a historical baseline illumination transfer matrix. Compared with the currently collected samples Solve for the local correction amount This allows the updated illumination transfer matrix to match the current sampled data. Using the currently running compensation parameters as the initial values ​​for iteration, a lightweight gradient descent algorithm is used to iterate 50 times to complete the correction calculation, with a total time of approximately 2 seconds.

[0213] (III) Parameter Hot Update

[0214] The corrected compensation parameters were uploaded to the GPU shader via the communication bus, overwriting the current operating parameters. During the parameter update process, the projected image continued to output without any interruption, flickering, or jumps, and the audience did not notice anything unusual. The system continued to run with the updated parameters until the end of the performance.

[0215] (iv) Multi-round triggering scenarios

[0216] In this embodiment, the system triggered incremental re-optimization twice during the 90-minute performance: the first time was when the sky darkened significantly in the evening (brightness trigger), and the second time was when cloud cover caused rapid fluctuations in ambient light (brightness + color temperature trigger). Both updates completed the test pattern projection and acquisition within 1 second, and the correction calculation and parameter hot update were completed within 2 seconds. The total background update time did not exceed 6 seconds, and the performance was not interrupted throughout.

[0217] In contrast, if the traditional method is used, the performance needs to be interrupted and all 27 color cards need to be re-acquired and fully calibrated if the ambient light changes beyond the acceptable range, which takes about 1 to 2 hours and cannot meet the continuous playback requirements of live performances.

[0218] This embodiment verifies that the method of the present invention can respond to dynamic changes such as ambient light fluctuations and color temperature shifts in outdoor live performance scenarios, and can complete adaptive correction without stopping the machine, thus meeting the engineering requirements for long-term stable operation.

[0219] The above description is only used to illustrate the technical solutions of the present invention and is not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention (such as the application of various formulas, the order of steps, etc.) without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain, characterized in that, Includes the following: Step S1, Data Acquisition and Preprocessing: Acquire raw point cloud data of the mountain, establish a global coordinate system and complete multiple sets of point cloud registration preprocessing, and output a standard point cloud dataset under a unified coordinate system. Step S2: Construct a digital twin model of the mountain projection area and complete material partitioning: Construct a three-dimensional mesh model of the mountain slope based on the standard point cloud dataset, solve the spatial parameters of non-node positions inside the mesh unit by bilinear interpolation, and solve the normal vector of the mountain slope in a unified manner; combine the real-world landform features and texture information of the mountain to complete the material region partitioning on the surface of the digital twin model and construct a mountain surface spectral reflectance database; Step S3: Establish the geometric-photometric coupling transfer model and the multi-projection illumination transfer matrix: Based on the three-dimensional coordinates and unit normal vector of the slope obtained from the digital twin model, and combined with the coordinates of the fixed installation points of each projector, solve the geometric optical influence factor of the mountain slope pixel by pixel; adopt the third-order Bernstein basis FFD free deformation model, and solve the inherent color transfer function of the projector through the multi-slope differential calibration mechanism. Based on the geometric optical influence factor, surface spectral reflectance and Gaussian point spread function, a multi-projection unified illumination transmission matrix is ​​constructed. Step S4, convex optimization to jointly solve the geometric photometric compensation parameters: Based on the target display screen and the illumination transfer matrix, pixel smoothing constraints and regularization constraints are introduced to construct a unified convex optimization objective function and solve the geometric deformation compensation parameters and color photometric compensation parameters. Step S5, Real-time Joint Correction Rendering Output: The obtained geometric deformation compensation parameters and color photometric compensation parameters are encapsulated into the pixel shader of the graphics processor, and an inverse mapping relationship is established from the original image pixels to the mountain projection slope pixels. The real-time input image is corrected and rendered frame by frame and then output. Step S6, Sensor Closed-Loop Incremental Adaptive Re-optimization: Real-time acquisition of on-site working condition data. When the ambient light brightness, color temperature, or projector light output status meets the preset trigger conditions, the incremental re-optimization process is automatically triggered, the compensation parameters are updated, and the data is synchronized to the graphics processor shader in real time.

2. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 1, characterized in that, The acquisition of raw point cloud data of the mountain in step S1 includes: A ground-based 3D laser scanner was used to scan the mountain slope from multiple stations to obtain high-precision topographic point clouds at close range. Aerial oblique photogrammetry using a drone equipped with an RTK positioning module was employed to acquire images of a wide range of mountain top surfaces and distant contours and generate airborne point clouds. No fewer than five ground feature control points with precise RTK coordinates are deployed on the ground in the projection area as coordinate system binding references; using the RTK ground feature control points as coordinate system binding references, coordinate unification and fusion of ground laser point cloud and UAV airborne point cloud are achieved.

3. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 1, characterized in that, The method for constructing the mountain surface spectral reflectance database in step S2 is as follows: By combining the real-world landform features and texture information of the mountain, material areas are divided on the surface of the digital twin model to distinguish different surface material areas such as rocks, bare soil, vegetation, and artificial structures. For each type of material region, at least 20 uniformly distributed sampling points are randomly selected, and RGB three-channel spectral reflectance data are obtained using a spectral acquisition device. The average value of the region is then used as the reference spectral reflectance for that type of material. For reflectance parameters at arbitrary coordinate positions of the model, a bicubic spline interpolation method is used to achieve continuous query. The spline interpolation boundary adopts natural boundary conditions, where the second derivative is zero, to avoid abnormal oscillations in the interpolation results at the material partition boundary, and to construct a comprehensive mountain surface spectral reflectance database.

4. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 1, characterized in that, Step S3, which describes solving the projector's inherent color transfer function using a multi-slope differential calibration mechanism, specifically includes: Select at least three hillside surfaces of different materials and background colors as calibration areas, and query the reflectance parameters of the corresponding locations on each calibration slope surface from the surface spectral reflectance database constructed in step S2. ; Twenty-seven standard solid color test charts were projected sequentially onto each calibrated slope, and real-time images of each slope were simultaneously captured by an industrial camera. And capture the background image of the slope under a completely black projection screen. ; The original image data collected from each slope were reverse-corrected, removing reflections from the mountain material, geometric light and shadow attenuation, and ambient background light interference pixel by pixel. The correction formula is as follows: in, Geometric optics influence factor; All slope-corrected sample data are merged into a global optimization sample set. A least-squares objective function with second-order difference smoothing regularization is constructed. The optimal control point parameters of the third-order Bernstein basis FFD model are solved to obtain the projector's inherent color transfer function. .

5. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 1, characterized in that, The multi-projection unified illumination transfer matrix mentioned in step S3 matrix elements Assign values ​​as follows: in, Geometric optics influence factor, , The angle between the projected ray and the slope normal. This is the spatial distance from the optical center of the projector to the point on the slope. For the slope Surface spectral reflectance at each sampling point; It is a two-dimensional Gaussian point spread function. , For the first The projector pixel corresponds to the center of the light spot and the slope surface. Spatial Euclidean distance between sampling points This is the diffusion radius parameter.

6. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 5, characterized in that, To satisfy pixel pairs, let This enables the sparse construction of the illumination transmission matrix.

7. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 6, characterized in that, The The truncation threshold is determined based on the following: In the two-dimensional Gaussian point spread function, the energy at a distance of 3σ from the center of the light spot has decayed to 0.1% of the central peak, meaning that most of the energy of the entire light spot is concentrated within the 3σ radius. After truncating the long-distance contribution with 3σ as the boundary, the storage complexity of the illumination transfer matrix decreases from... Down to Where K is the number of effective influencing pixels of a single projector pixel within the camera's field of view, and K is less than N. c The 10% loss corresponds to a modeling accuracy loss of less than 1.1%.

8. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 1, characterized in that, The convex optimization objective function mentioned in step S4 is: in, This is the optimal projection compensation parameter matrix; Display the screen for the target; For illumination transfer matrix; It is the set of all adjacent pixel pairs; Adjacent pixels and The compensation parameter vector; Smoothing weights for adjacent pixels; is the Tikhonov regularization coefficient.

9. The method for joint adaptive correction of multi-projection geometric photometry in irregular mountainous terrain according to claim 1, characterized in that, The preset triggering conditions in step S6 are: ambient light brightness fluctuation exceeds the reference value by ±15%, color temperature shift exceeds ±500K, or projector light output attenuation is greater than 5%; in the incremental re-optimization process, one set of full white uniform field pattern and one set of grayscale gradient pattern are projected, the local correction amount is solved based on the historical reference illumination transmission matrix and the current collected sample, and the update is completed by gradient descent with 50 iterations.

10. A system for implementing the method according to any one of claims 1-9, characterized in that, It includes a multi-source data acquisition and point cloud preprocessing module, which is used to acquire raw point cloud data of the mountain and perform preprocessing and coordinate registration and fusion, and output standard mountain elevation point cloud data under a unified coordinate system; The digital twin modeling and material partitioning module is used to construct a three-dimensional mesh model of the mountain slope based on the point cloud data, solve the spatial parameters of non-node positions inside the mesh unit by bilinear interpolation, and construct a database of spectral reflectance of the mountain surface. The geometry-photometric coupling transfer model construction module is used to solve the three-dimensional coordinates and unit normal vector of the slope based on the digital twin model, calculate the geometric optical influence factor for each projected pixel, solve the inherent color transfer function of the projector through the multi-slope differential calibration mechanism, and construct a unified illumination transfer matrix for multiple projections based on the geometric optical influence factor, surface spectral reflectance and Gaussian point spread function. The convex optimization joint solution module is used to construct a unified convex optimization objective function based on the target display screen and the illumination transmission matrix, and simultaneously solve for the geometric deformation compensation parameters and color photometric compensation parameters. The real-time correction rendering output module is used to encapsulate the compensation parameters into the graphics processor pixel shader, establish the inverse mapping relationship between the original image pixels and the mountain projection slope pixels, perform frame-by-frame correction rendering on the real-time input screen and output it to each engineering projector. The sensor closed-loop incremental adaptive re-optimization module is used to collect on-site working condition data in real time. When the ambient light brightness, color temperature or projector light output status meets the preset trigger conditions, the incremental re-optimization process is automatically triggered to update the compensation parameters and synchronize them to the graphics processor shader in real time.

Images