Campus non-standard wall surface three-dimensional reconstruction and parameter calibration method and system

Abstract

This application relates to the field of computer vision and 3D reconstruction technology, and provides a method and system for 3D reconstruction and parameter calibration of non-standard campus walls. The method includes: acquiring image data of the target scene and corresponding environmental state data; parsing a set of implicit correlation parameter models between the environmental state data and image features, wherein the set of implicit correlation parameter models includes at least one of a nonlinear mapping model, a statistical correlation model, and a correlation model between light source direction and scene shadow distribution; adjusting at least one of the feature extraction parameters and geometric constraint weights in the 3D reconstruction process based on the set of implicit correlation parameter models; and performing 3D reconstruction on the image data based on the adjusted feature extraction parameters and geometric constraint weights to obtain a 3D model of the target scene. This application achieves adaptive high-precision 3D reconstruction based on multi-factor joint analysis, improving the stability and accuracy of reconstruction in complex environments.

Description

Technical Field

[0001] This application relates to the field of computer vision and 3D reconstruction technology, and in particular to a method and system for 3D reconstruction and parameter calibration of non-standard campus walls. Background Technology

[0002] Currently, in the field of campus digital twin and smart campus construction, the three-dimensional reconstruction of building facades usually adopts a technical approach that combines motion recovery structure and multi-view stereo vision. This approach extracts image feature points and performs multi-view matching to restore the geometric structure and texture information of the scene.

[0003] During actual campus non-standard wall surface acquisition and reconstruction operations, due to the highly dynamic nature of the outdoor environment, technicians often find the following issues during inspection: the distribution of image feature points on the same wall surface shifts significantly under different lighting conditions, such as morning side lighting and midday overhead lighting, and descriptors differ greatly, leading to a sharp drop in cross-time feature matching success rate; under different lighting conditions, the acquired textures exhibit visible color differences and brightness jumps after mapping; seasonal vegetation or temporary construction obstructions cause image connectivity breaks over time; and non-standard geometric shapes such as curved surfaces and irregular decorations show reconstruction deviations under fixed constraints. Analysis of these problems reveals that the root cause lies in the fact that existing 3D reconstruction technologies treat image data as an isolated information source, failing to establish an inherent mapping mechanism between environmental state data (such as time series, lighting, and weather) and image feature representation. This results in the reconstruction algorithm's inability to perceive and adapt to dynamic changes in external environmental conditions, leading to a lack of lighting priors in feature extraction, a lack of dynamic adjustment of geometric constraints, and a lack of spatiotemporal models for occlusion handling. Ultimately, this results in poor stability and insufficient accuracy of the reconstruction system in complex and variable environments. Summary of the Invention

[0004] To address the problem that the inherent correlation between environmental state data and image features in existing technologies is not effectively mined and utilized, resulting in the inability of 3D reconstruction systems to adapt to environmental changes, this application proposes a method and system for 3D reconstruction and parameter calibration of non-standard campus walls, which achieves adaptive high-precision 3D reconstruction based on multi-factor joint analysis.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, a method for 3D reconstruction and parameter calibration of non-standard campus walls includes: acquiring image data of the target scene and corresponding environmental state data; analyzing a set of implicit correlation parameter models between the environmental state data and image features, wherein the implicit correlations include at least one of the following: a nonlinear mapping model between ambient light intensity and global image brightness, a statistical correlation model between ambient light intensity and feature point detection repetition rate, and a correlation model between light source direction and scene shadow distribution, used to characterize the influence of different environmental states on image feature representation; adjusting at least one of the feature extraction parameters and geometric constraint weights in the 3D reconstruction process based on the set of implicit correlation parameter models; and performing 3D reconstruction on the image data based on the adjusted feature extraction parameters and geometric constraint weights to obtain a 3D model of the target scene.

[0006] In conjunction with the first aspect mentioned above, in one possible implementation, the construction process of the implicit correlation parameter model set includes: acquiring historically collected multidimensional time-series data, including historical environmental state data and historical image feature data aligned to timestamps; reconstructing the phase space of the multidimensional time-series data and generating the reconstructed phase space trajectory using the delayed coordinate embedding method; using a convergent cross-mapping algorithm to test the causal relationship of the phase space trajectory, and determining the causal driving direction and strength between historical environmental state data and historical image feature data by calculating the cross-mapping prediction error; using dynamic factor analysis to extract potential common driving factors from multidimensional variables with causal driving direction and strength, used to decouple the coupling effects under complex environmental conditions; and quantifying the mathematical expressions of nonlinear mapping models, statistical correlation models, and correlation models based on the causal driving direction and strength and common driving factors, assigning a confidence score to each model to form a parameterized model set, with the confidence score allocation based on statistical significance and causal test strength.

[0007] In conjunction with the first aspect mentioned above, one possible implementation involves adjusting the feature extraction parameters during 3D reconstruction based on a set of latent correlation parameter models. This includes: calculating the optimal feature detection threshold based on latent correlations, according to the ambient light intensity at the current image acquisition time. During the feature point detection stage, the optimal feature detection threshold is applied to the feature point detection algorithm. During the descriptor calculation stage, environmental state data is encoded as conditional input to constrain the consistency of the generated feature point descriptors under different lighting conditions.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, during the descriptor computation stage, environmental state data is encoded as conditional input to constrain the consistency of the generated feature point descriptors under different lighting conditions. This process includes: extracting ambient light intensity and light source orientation angle from the environmental state data, and mapping these parameters into a high-dimensional environmental state conditional vector using a multilayer perceptron network. In the descriptor generation network, initial feature vectors of local image regions are extracted through convolutional layers. The environmental state conditional vector and the initial feature vector are concatenated along the channel dimension to obtain a fused feature vector. The fused feature vector is input into the conditional adaptive normalization layer of the descriptor generation network. Using the environmental state conditional vector, scaling and translation parameters are calculated through fully connected layers to perform an affine transformation on the fused feature vector to calibrate the feature distribution. Based on the affine transformed feature vector, an illumination-adaptive feature point descriptor is generated to eliminate the phenomenon of increased descriptor distance caused by changes in ambient light.

[0009] In conjunction with the first aspect mentioned above, in one possible implementation, the process of adjusting the geometric constraint weights during 3D reconstruction based on a set of latent correlation parameter models includes: obtaining the classification results and geometric parameters of non-standard geometric shapes in the target scene. These non-standard geometric shapes include at least one of the following: curved walls, irregular decorative surfaces, stepped facades, and inclined support columns. The geometric parameters include at least the radius of curvature of the curved area, the height difference of the stepped surfaces, and the tilt angle of the facade. Based on the classification results and geometric parameters of the non-standard geometric shapes, the geometric type of the area to be reconstructed in the target scene and the corresponding initial constraint smoothness parameters are determined. Based on the solar altitude angle and wall orientation at the current image acquisition time, the correlation model between the light source direction and the scene shadow distribution in the latent correlation parameter model set is invoked to calculate the shadow distribution characteristics of the area to be reconstructed under the current lighting conditions. The shadow distribution characteristics include at least the shadow coverage and the shadow boundary position. Based on the shadow distribution characteristics, the initial constraint smoothness parameters are adjusted to generate optimized constraint smoothness parameters that match the current lighting conditions and non-standard geometric shapes. Based on the optimized constraint smoothness parameters, surface fitting and surface optimization are performed on the point cloud data of the area to be reconstructed to generate a reconstructed surface with non-standard geometric shapes.

[0010] In conjunction with the first aspect described above, one possible implementation further includes: identifying curved regions in the target scene and extracting the radius of curvature of the curved regions. Based on the radius of curvature and the current lighting conditions, the smoothness parameter for surface fitting is adjusted. In response to a radius of curvature less than a preset curvature threshold and under side lighting conditions, the smoothness parameter is reduced to preserve surface detail features, and the weight of line segment constraints is increased to constrain the fitting direction of the curved region. In response to a radius of curvature greater than or equal to a preset curvature threshold and under direct top lighting conditions, the weights of the smoothness parameter and the surface normal constraint are increased to suppress surface undulations caused by noise and ensure the continuity of the surface normal. Based on the dynamically adjusted smoothness parameter, the weights of the line segment constraints, and the weights of the surface normal constraints, point cloud optimization and surface fitting are performed on the curved regions.

[0011] In conjunction with the first aspect mentioned above, one possible implementation also includes establishing spatiotemporal labels for occluders in the target scene, including their occurrence time and impact level. During the feature matching stage, the matching weight of feature points in the occluded area is reduced based on the spatiotemporal labels. In the multi-view fusion stage, complementary image information collected at different times is used to compensate and reconstruct the occluded area.

[0012] In conjunction with the first aspect mentioned above, in one possible implementation, the process of compensating and reconstructing occluded areas using complementary image information acquired at different time periods during the multi-view fusion stage includes: acquiring image sequences acquired from various perspectives and at different time periods in the target scene, as well as the acquisition timestamps and spatiotemporal labels corresponding to each image. The spatiotemporal labels at least include the type of occlusion, the time period of occurrence, and the degree of impact. Based on the spatiotemporal labels, occlusion areas are labeled in each image of the image sequence to generate an occlusion mask map, which is used to mark the pixel areas covered by occluded objects in the image. Using the 3D model of the target scene as a spatial reference, the images acquired at different time periods are projected onto the corresponding surface areas of the 3D model, and based on the occlusion mask map, the visibility status of each surface area in each image is determined. The visibility status includes visible, occluded, and partially visible. For target surface areas marked as occluded in the first image acquired at the first time period and marked as visible in the second image acquired at the second time period, the texture information and geometric constraint information of the corresponding areas in the second image are extracted. The texture information and geometric constraint information are mapped to the corresponding positions of the target surface region in the 3D model according to the projection mapping relationship, generating compensation texture and compensation geometric data. Based on this, surface filling and smoothing optimization are performed on the target surface region to obtain the 3D model after occlusion compensation.

[0013] Secondly, a system for 3D reconstruction and parameter calibration of non-standard campus walls is provided, comprising: a data acquisition module for acquiring image data of the target scene and corresponding environmental state data; a correlation analysis module for analyzing a set of implicit correlation parameter models between environmental state data and image features, wherein the implicit correlations include at least one of the following: a nonlinear mapping model between ambient light intensity and global image brightness, a statistical correlation model between ambient light intensity and feature point detection repetition rate, and a correlation model between light source direction and scene shadow distribution, used to characterize the influence of different environmental states on image feature representation; a parameter dynamic adjustment module for adjusting at least one of the feature extraction parameters and geometric constraint weights in the 3D reconstruction process based on the set of implicit correlation parameter models; and a 3D reconstruction module for performing 3D reconstruction on the image data based on at least one of the adjusted feature extraction parameters and geometric constraint weights to obtain a 3D model of the target scene.

[0014] In conjunction with the second aspect described above, one possible implementation also includes an illumination adaptive calibration and correction module. This module is used to: synchronously record environmental state data in response to a camera calibration start signal; evaluate the reliability of corner detection on the calibration board based on a set of latent correlation parameter models; and, in response to a calibration result optimization start signal, remove or reduce the weight of low-reliability calibration images, and output camera calibration parameters with confidence evaluation.

[0015] This application provides a method and system for 3D reconstruction and parameter calibration of non-standard campus walls. It can transform dynamic environmental information such as illumination, time series, and weather into computable mathematical models by analyzing the implicit correlation parameter model set between environmental state data and image features. This serves as dynamic prior knowledge for the 3D reconstruction algorithm, enabling the reconstruction system to perceive and adapt to environmental changes. Furthermore, an environmental state condition vector is introduced during the descriptor calculation stage. An adaptive normalization layer is used to calibrate the feature distribution through affine transformation, achieving conditional decoupling of the feature descriptor from environmental illumination. This ensures that the generated descriptors for the same physical point maintain consistency in the feature space under different illumination conditions, effectively eliminating the phenomenon of increased descriptor distance caused by illumination changes and improving the feature matching success rate across time periods and illumination conditions. Simultaneously, by establishing a correlation model between the light source direction and scene shadow distribution, and dynamically adjusting the geometric constraint weights in conjunction with non-standard geometric morphology parameters, shadows are used to enhance line segment constraints to preserve details under side lighting conditions, while normal constraints are enhanced under top lighting conditions to ensure continuity. This overcomes the systematic biases generated by traditional fixed constraint methods when processing non-standard geometric morphologies such as curved surfaces and irregular decorations, improving geometric fidelity. In addition, spatiotemporal labels containing the time of occurrence and degree of impact are established for occlusions. During the multi-view fusion stage, complementary image information from different time periods is used for compensation and reconstruction, transforming the occlusion processing from passive removal to active spatiotemporal complementation. This effectively solves the problems of image connection breaks and model holes caused by occlusions such as vegetation and temporary facilities, ensuring the integrity of the 3D model and ultimately improving the stability and accuracy of reconstruction in complex and ever-changing campus environments. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a method for three-dimensional reconstruction and parameter calibration of non-standard campus walls provided in this application embodiment; Figure 2 The flowchart for constructing the implicitly related parameter model set in a method for three-dimensional reconstruction and parameter calibration of non-standard campus walls provided in this application embodiment; Figure 3 The flowchart of illumination adaptive feature extraction and descriptor generation in a method for three-dimensional reconstruction and parameter calibration of non-standard campus walls provided in this application embodiment; Figure 4 A flowchart illustrating the dynamic adjustment of geometric constraint weights in a method for three-dimensional reconstruction and parameter calibration of non-standard campus walls provided in this application embodiment; Figure 5 A detailed flowchart of adaptive fitting of curved surface region in a method for three-dimensional reconstruction and parameter calibration of non-standard campus walls provided in this application embodiment; Figure 6 A flowchart of dynamic occlusion spatiotemporal label establishment and compensation reconstruction in a method for three-dimensional reconstruction and parameter calibration of non-standard campus walls provided in this application embodiment; Figure 7 A detailed flowchart of occlusion compensation reconstruction in a method for three-dimensional reconstruction and parameter calibration of non-standard campus walls provided in this application embodiment; Figure 8 This is a schematic diagram of a three-dimensional reconstruction and parameter calibration system for non-standard campus walls provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0019] Example 1: like Figure 1 As shown, this embodiment provides a method for 3D reconstruction and parameter calibration of non-standard campus walls. This method achieves adaptive high-precision 3D reconstruction in complex and ever-changing campus environments by constructing a latent correlation driving mechanism between environmental states and image features. The method flow of this embodiment mainly includes the following steps: Step S100: Obtain image data of the target scene and corresponding environmental state data.

[0020] The target scene specifically refers to the exterior facade area of ​​campus buildings that includes non-standard geometric forms such as curved walls, irregular decorative surfaces, and stepped facades.

[0021] Image data refers to the original image sequence obtained by capturing a target scene from multiple angles using multi-view cameras (such as drone tilt cameras or ground close-up cameras). Each image carries a capture timestamp accurate to the second.

[0022] Environmental condition data refers to a set of multidimensional parameters that are recorded synchronously at the moment of image acquisition and can characterize the external physical environment conditions. It includes not only optical parameters such as ambient light intensity, solar altitude angle, and solar azimuth angle, but also spatiotemporal semantic information such as weather type (sunny, cloudy, rainy, foggy), season, and time of day, as well as other environmental factors such as temperature, humidity, and wind speed that may affect image quality or scene appearance.

[0023] In some implementations, hardware trigger signals or software timestamp alignment mechanisms are used to ensure that each frame of an image has a unique corresponding environmental state record at the moment of acquisition. For example, when a drone with a camera takes a picture of a wall, the onboard sensors or ground base station simultaneously record the illuminance meter reading, GPS location, and time information at that moment, and query the detailed meteorological data at that moment through a network API, packaging and storing these heterogeneous data into an aligned sample record.

[0024] It is worth noting that this step, through the simultaneous acquisition of multi-source data, breaks through the limitations of traditional 3D reconstruction schemes that rely solely on images as a single information source. It incorporates environmental conditions as equally important primary data into the reconstruction process, providing a solid data foundation for subsequent exploration of the deep physical laws governing the relationship between the environment and features. This allows the reconstruction system to perceive environmental changes at the source, rather than passively responding to image quality degradation. Simultaneously, the synchronous acquisition mechanism ensures a one-to-one correspondence between environmental variables and image representation in subsequent analysis, avoiding spurious correlations caused by temporal misalignment.

[0025] Step S200: Analyze the set of implicit correlation parameter models between environmental state data and image features. The implicit correlation relationship includes at least one of the following: a nonlinear mapping model between ambient light intensity and global image brightness, a statistical correlation model between ambient light intensity and feature point detection repetition rate, and a correlation model between light source direction and scene shadow distribution. These models are used to characterize the influence of different environmental states on image feature performance.

[0026] The implicit correlation parameter model set consists of pre-constructed or offline-trained mathematical models or parameterized functions. Its implicit correlations are nonlinear patterns automatically discovered from historical observation data using data mining algorithms, reflecting the inherent causal driving relationship between environmental variables and image features. For example, the relationship between ambient light intensity and global image brightness is not a simple direct proportionality, but a complex nonlinear mapping influenced by factors such as wall material reflectivity and atmospheric scattering; the relationship between light source direction and scene shadow distribution also exhibits spatial transformation patterns dependent on building geometry. This model set can take various forms, such as neural network models, polynomial fitting functions, Gaussian process regression models, or high-dimensional lookup tables.

[0027] In some implementations, before deployment, the system utilizes a large amount of historically collected aligned data (i.e., historical samples containing environmental states and image features) and causal discovery algorithms such as phase space reconstruction and convergent cross mapping to discover stable hidden correlation patterns and solidify them into model parameters.

[0028] During actual reconstruction operations, the system takes the currently collected environmental state data as input and feeds it into the model set. The model immediately outputs the expected image feature parameters (such as expected brightness distribution, expected feature point density, expected shadow mask, etc.) under the current environment. This forms a mechanism that combines pre-construction and real-time parsing, ensuring the scientific accuracy of the correlation while avoiding the huge computational overhead of online learning, thus meeting the real-time requirements of 3D reconstruction.

[0029] It should be noted that this step, by introducing a set of implicit correlation parameter models, transforms abstract environmental physical quantities into computable prior knowledge of image features. This enables the reconstruction algorithm to no longer blindly process images, but to "predict" what features the image should present under the current environmental conditions, providing deterministic guidance for subsequent parameter adjustments.

[0030] Step S300: Based on the set of implicit correlation parameter models, adjust at least one of the feature extraction parameters and geometric constraint weights in the 3D reconstruction process.

[0031] The adjustment process refers to the dynamic correction of key control parameters within the 3D reconstruction algorithm based on the model output obtained from step S200. This adjustment is an adaptive behavior based on deterministic mapping relationships, rather than random trial and error or static configuration based on human experience.

[0032] Feature extraction parameters include, but are not limited to, contrast threshold, edge response threshold, and non-maximum suppression radius for feature point detection. These parameters directly determine the number, quality, and distribution of feature points extracted from the image.

[0033] Geometric constraint weights refer to the coefficients used to balance the influence of data terms and regularization terms in stages such as surface fitting and point cloud optimization. Examples include line segment constraint weights, normal consistency weights, and smoothness weights. They determine the degree to which the reconstructed surface follows geometric priors.

[0034] In some implementations, when the model output indicates that the current environment is under strong light and overexposure, the system automatically increases the feature detection threshold to filter out false high-frequency noise points caused by overexposure, while reducing the weight of the descriptor dimension that is sensitive to brightness. When the model output indicates that the current environment is under side lighting and there is obvious projection on the wall, the system automatically increases the weight of line segment constraints, because at this time the shadow boundary enhances the visibility of the building structure lines, and the line segment constraints can more accurately restore the wall geometry. Conversely, if the environment is under midday top lighting and the projection disappears, the weight of the surface normal constraint is increased, relying on surface continuity to suppress noise.

[0035] It should be noted that this step injects prior environmental knowledge into the algorithm's internal control loop, enabling feature extraction and geometric optimization to dynamically adjust according to environmental conditions, always maintaining optimal operating status under current conditions. This deterministic adjustment mechanism based on the implicit association model not only avoids the efficiency problem of heuristic search but also solves the problems of feature matching failure and geometric reconstruction distortion caused by environmental mismatch.

[0036] Step S400: Based on at least one of the adjusted feature extraction parameters and geometric constraint weights, perform three-dimensional reconstruction on the image data to obtain a three-dimensional model of the target scene.

[0037] 3D reconstruction refers to the complete processing flow of recovering the 3D geometric structure and texture information of a scene from a 2D image sequence using computer vision algorithms. In this embodiment, the process does not run in isolation but is deeply integrated with dynamically adjusted parameter configurations from previous steps. Specifically, the feature matching stage uses adjusted thresholds to extract key points and generate descriptors; the pose estimation stage uses high-quality matching point pairs to solve for camera motion; and the dense reconstruction and surface fitting stage applies dynamically adjusted geometric constraint weights to optimize point cloud and mesh generation. The final output 3D model not only contains accurate geometric topology but also has consistent texture mapping, enabling it to realistically reproduce the detailed features of non-standard campus walls.

[0038] In some implementations, the 3D reconstruction process encompasses a fully adaptive end-to-end process, from sparse to dense reconstruction. For example, in the feature matching stage, the use of environment-adaptive feature extraction parameters improves the matching success rate across time periods and illumination levels, ensuring the stability of camera pose estimation. In the surface fitting stage, because the geometric constraint weights dynamically change with lighting and geometric shape, the curved wall retains more details under side lighting and is smoother and more continuous under top lighting, effectively suppressing the "staircase effect" and noise fluctuations. In addition, during the reconstruction process, texture color normalization can be performed by combining environmental state data to further eliminate color differences.

[0039] It should be noted that this step, through end-to-end parameter adaptation, successfully overcame the unique challenges of temporal lighting variations, dynamic occlusion interference, and non-standard geometric deviations inherent in the campus environment, generating a high-precision, highly complete 3D model. This not only validates the effectiveness of the implicit correlation-driven reconstruction strategy but also provides a reliable spatial data foundation for subsequent digital twin applications, ancient building preservation, and smart campus management.

[0040] Based on the above technical solutions, this embodiment establishes a new paradigm of 3D reconstruction with the implicit correlation between environment and features as the core driving force by constructing a complete method framework for environmental state acquisition, implicit correlation parsing, dynamic parameter adjustment and adaptive reconstruction. It deeply integrates the originally fragmented environmental information with the image processing process, enabling the reconstruction system to have environmental adaptability similar to human vision and solving the problem of performance degradation of fixed parameter algorithms in complex dynamic scenes.

[0041] Example 2: like Figure 2 As shown, this embodiment, based on Embodiment 1, further elaborates on the specific construction process of the implicit correlation parameter model set. This construction process aims to automatically mine the deep-seated causal driving patterns between environmental variables and image features from historical data, rather than merely remaining at the surface statistical correlation level, thereby providing a physically interpretable prior knowledge foundation for subsequent adaptive reconstruction. The construction process of this embodiment mainly includes the following steps: Step S201: Obtain historically collected multidimensional time series data, which includes historical environmental state data aligned by timestamps and historical image feature data.

[0042] Multidimensional time-series data refers to a structured dataset formed by precisely aligning heterogeneous environmental sensor data and image feature data on a unified time axis. Its historical environmental state data includes not only instantaneous physical quantities such as light intensity and solar altitude angle, but also derived features such as the preprocessed temporal change rate; historical image feature data covers quantitative indicators that characterize image imaging quality, such as global brightness mean, gradient entropy, and feature point response intensity distribution.

[0043] It should be understood that the "alignment" here is not a simple timestamp matching, but a spatiotemporal synchronization calibration that takes into account the differences in sensor sampling frequencies and data transmission delays, ensuring that each set of data samples truly reflects the correspondence between environmental conditions and imaging results at the same moment.

[0044] In some implementations, the system first imports raw data in batches from campus environmental monitoring stations, meteorological API interfaces, and camera logs, and then performs outlier removal and missing value imputation operations: For outliers caused by sensor malfunctions, a median filter based on a sliding window is used for smoothing. For short-term data gaps, cubic spline interpolation or linear extrapolation of data from adjacent time points can be used to fill the gaps.

[0045] Based on this, using the image acquisition timestamp as the primary key, the environmental data is resampled to a time grid consistent with the image frame rate, ultimately constructing a structure resembling... The aligned dataset, where This represents the environment state vector at time t. Let represent the image feature vector at time t, and N be the total number of samples. This strict data alignment mechanism eliminates spurious correlation noise introduced by temporal misalignment, providing a high-quality data foundation for subsequent nonlinear dynamic analysis.

[0046] It should be noted that this step solves the problem of joint analysis of multi-source heterogeneous data by constructing standardized multi-dimensional time series data. In a real campus environment, meteorological data is usually updated on a minute or even hourly basis, while image acquisition may be triggered on a second-by-second basis, resulting in a natural scale mismatch. This step, through refined alignment and preprocessing, transforms discrete observation points into continuous dynamic evolution trajectories, thereby elevating the originally isolated environment and image pairing data to a time-series manifold containing the evolutionary laws of the system.

[0047] Step S202: Reconstruct the phase space of the multidimensional time series data and generate the reconstructed phase space trajectory by using the delayed coordinate embedding method.

[0048] Phase space reconstruction is a nonlinear time series analysis method based on Takens' theorem. It mainly involves selecting appropriate embedding dimensions and delay times to map one-dimensional or multi-dimensional time series to a high-dimensional phase space, so that the reconstructed phase space trajectory is topologically equivalent to the attractor of the prime mover system.

[0049] In this embodiment, the delayed coordinate embedding method is to embed the scalar time series... Transform into a high-dimensional vector sequence The process, in which For the embedding dimension, The time delay is used to generate the phase space trajectory. The generated phase space trajectory retains the geometric structure and invariant characteristics of the original system's dynamic evolution, enabling the recovery of the hidden state variable evolution law based solely on observational data in the absence of system analytical equations.

[0050] In some implementations, the dimension is embedded. and delay time The choice of latency directly determines the quality of the reconstruction. Specifically, for latency... The mutual information method is used to find the first local minimum point to ensure that the delayed coordinate components are neither excessively redundant nor completely unrelated, thus maximizing the expansion of the attractor structure. For embedding dimension The dimensionality is gradually increased using a pseudo-nearest neighbor method until the proportion of pseudo-nearest neighbors drops below a preset threshold (e.g., 5%). At this point, the attractor is considered to be fully expanded and without self-overlap. For example, when processing the joint temporal sequence of illumination intensity and image brightness, if the optimal... =3 (corresponding to 3 sampling intervals) If the value is 5, the original two-dimensional time series will be reconstructed into a 10-dimensional phase space trajectory vector.

[0051] It should be noted that existing technologies typically establish a mapping between the environment and the image directly in the original pixel space or shallow feature space, ignoring the fact that the data acquisition process is essentially a dynamic system evolution process governed by physical laws. This step introduces phase space reconstruction technology to restore the seemingly chaotic time-series observation data into geometric objects with deterministic structures. This allows subsequent analysis to no longer rely on fragile statistical assumptions, but rather to be based on a solid foundation of nonlinear dynamics theory, thereby improving the ability to capture the changing patterns of complex environments.

[0052] Step S203: Use the convergent cross-mapping algorithm to test the causal relationship of the phase space trajectory. By calculating the cross-mapping prediction error, determine the causal driving direction and strength between historical environmental state data and historical image feature data.

[0053] Converging cross-mapping is an algorithm used to detect causal relationships in nonlinear coupled systems. Its basic principle is: if variable X causally drives variable Y, then the information of X is encoded into the time series of Y. Therefore, the state of X can be accurately predicted through the reconstructed phase space (shadow manifold) of Y, and this prediction accuracy converges as the length of the shadow manifold increases. Conversely, if there is only correlation without causality, or if the causal directions are opposite, this convergence phenomenon will not occur. In this embodiment, the cross-mapping prediction error is used as an indicator to quantify the strength of causality; the smaller the error and the more obvious the convergence trend, the stronger the causal driving force.

[0054] In some implementations, the system constructs shadow manifolds for the environment state variable E. and the shadow manifold of image feature variable I First, using The value of E is predicted by using the neighboring point weights, thus obtaining the predicted sequence. And calculate its correlation coefficient with the true E. Then, gradually increase the amount used for refactoring. The time series length L, observation The curve showing the change. If If the value shows a significant upward trend with increasing L and eventually stabilizes, then the statement "I causally drives E" is invalid, but "E causally drives I" may be valid (reverse verification is required). Specifically, when verifying "E drives I", if... After L reaches the critical length, it converges to above 0.8, while the reverse prediction If the correlation coefficient remains below 0.3 and does not converge, it can be confirmed that there is a unidirectional causal drive from the environment to the image, and the driving strength can be characterized by the correlation coefficient value after convergence, which can effectively eliminate spurious causal interference caused by seasonal periodic synchronous changes.

[0055] It should be noted that in outdoor campus scenes, the relationship between illumination and image features is highly nonlinear and coupled by multiple factors, making simple linear regression or Pearson correlation prone to misjudgment. CCM, by probing the topological mapping relationships between manifolds, can identify the true physical causal chains from pure observational data, ensuring that the subsequently established implicit correlation model reflects objective physical laws rather than data coincidences. This is the fundamental guarantee for improving the robustness of the reconstruction algorithm.

[0056] Step S204: Dynamic factor analysis is used to extract potential common driving factors from multidimensional variables with causal driving direction and intensity, which are used to decouple the coupling effects under complex environmental conditions.

[0057] Dynamic Factor Analysis (DFA) extracts a few unobservable latent common factors from time-series data of multiple observed variables. These factors represent independent physical mechanisms driving the evolution of the system. In this embodiment, since the environmental state itself contains multiple interrelated variables such as illumination, temperature, and humidity, their influence on image features is often coupled and superimposed. DFA can decompose these mixed signals into independent "common driving factors," such as separating the "solar radiation factor" from the "atmospheric scattering factor," thereby achieving a refined analysis of the impact of complex environments.

[0058] In some implementations, the system uses a subset of variables identified in step S203 as input to the DFA (Distributed Aspect-Oriented Analytical Formula). Specifically, a state-space model is constructed, assuming that the observation vector is a linear combination of a small number of latent factors through a loading matrix, plus a specific error term. Factor paths and model parameters are estimated jointly using Kalman filtering and the expectation-maximization (EM) algorithm. For example, if the input includes variables such as illumination intensity, color temperature, cloud cover, image brightness, and contrast, the DFA may extract two main factors: factor F1, with high loadings on illumination intensity and image brightness, is interpreted as the "direct light illumination factor"; factor F2, with high loadings on color temperature and image hue, and an increased variance contribution during cloudy periods, is interpreted as the "environmental diffuse reflection factor." This decoupling allows subsequent modeling to establish mapping relationships for independent physical mechanisms, avoiding model instability caused by multivariate collinearity.

[0059] It should be noted that in practical applications, modeling solely based on original environmental variables often faces severe multicollinearity problems, leading to difficult-to-interpret model parameters and poor generalization ability. The common driving factors extracted by DFA are not only statistically orthogonal, but also physically correspond to independent optical processes. This gives the implicit association model clear physical semantics, making it easier for technicians to understand and debug, while also significantly reducing model complexity and the risk of overfitting.

[0060] Step S205: Based on the causal driving direction and intensity and common driving factors, quantify and generate mathematical expressions for nonlinear mapping models, statistical correlation models and correlation models, and assign a confidence score to each model to form a set of parameterized models. The assignment of confidence scores is based on statistical significance and causal test strength.

[0061] The quantitative generation of mathematical expressions refers to the process of transforming the qualitative causal relationships and decoupled factors discovered in the preceding steps into computational functions that can be used for real-time inference. These functions can take the form of multinomial regression, Gaussian processes, neural networks, or lookup tables, depending on the distribution characteristics of the data.

[0062] The confidence score is a quantitative measure of the reliability of each generated model. It combines the p-value of the statistical test, the correlation coefficient after CCM convergence, and the prediction accuracy of cross-validation. It is used to guide the priority of model calls or weighted fusion strategies in the subsequent reconstruction process.

[0063] In some implementations, for each given pair of causal variables or factors and features, the system automatically tries multiple function fitting forms and selects the optimal model using the Bayesian Information Criterion (BIC).

[0064] Confidence score The following is an example of the calculation logic: ,in The p-value represents the statistical significance of the fitted residuals. The CCM convergence correlation coefficient obtained in step S203. The coefficient of determination for leave-one-out cross-validation. Set the preset weights (e.g., 0.3, 0.4, 0.3).

[0065] For example, in the "direct light illumination factor - image brightness" model, if , , Then the calculation yields The model is marked as a high-confidence model; however, for the "humidity-feature point repetition rate" model, if all indicators are weak, the calculated... In subsequent use, these models will be given lower weights or used only as a fallback when no other models are available. All models and their scores are ultimately encapsulated into a standardized set of parameterized models for use by downstream modules.

[0066] It should be noted that this step, through an explicit confidence scoring mechanism, endows the reconstruction system with a "metacognitive" ability regarding the reliability of its own knowledge. It mathematically quantifies the uncertainty differences in the association between different environments and features, enabling the system to automatically avoid being misled by low-confidence models when facing extreme or rare environmental conditions, and instead rely on more robust high-confidence channels. This significantly improves the system's safety and fault tolerance under unknown conditions while ensuring overall accuracy.

[0067] Based on the above technical solutions, this embodiment establishes a physically interpretable implicit correlation model generation methodology through a complete construction chain of data alignment, phase space reconstruction, causal testing, factor decoupling and model quantization, thereby breaking through the limitations of traditional computer vision fields that only focus on the image itself or simple statistical correlations.

[0068] Example 3: like Figure 3 As shown, this embodiment, based on Embodiment 1, further elaborates on the specific implementation method of adjusting feature extraction parameters in the 3D reconstruction process based on the implicit correlation parameter model set. This embodiment focuses on solving the problems of unstable feature point detection and descriptor matching failure caused by drastic changes in ambient lighting. By deeply embedding environmental state information into the entire link of feature extraction and descriptor generation, highly robust feature representation adapted to lighting is achieved. The adjustment process of this embodiment mainly includes the following steps: Step S301: Calculate the optimal feature detection threshold based on the ambient light intensity at the current image acquisition time and the implicit correlation.

[0069] The optimal feature detection threshold refers to a dynamic control parameter that enables the feature point detection algorithm to achieve the best balance between repetition rate and accuracy under specific ambient lighting conditions. This threshold is not a fixed value preset by humans, nor is it a simple linear proportional coefficient. Instead, it is a nonlinear mapping value obtained through real-time inference from the "Ambient Light Intensity-Feature Point Detection Repetition Rate Statistical Correlation Model" in the implicit correlation parameter model set constructed in Example 2. Its physical meaning is: the minimum response intensity boundary that can effectively distinguish real structural features from background noise under the current lighting intensity and corresponding image signal-to-noise ratio level.

[0070] In some implementations, the system first obtains the ambient light intensity at the current image acquisition time. Input it into a pre-trained statistical association model The model can be a nonlinear function based on Gaussian process regression or neural network fitting, and its output is the optimal feature detection threshold. The specific calculation logic can be expressed as follows: ,in These are the model parameters.

[0071] For example, when the ambient light intensity increases from 5,000 lux in the early morning to 80,000 lux at noon, the strong light causes local overexposure of the wall surface and gradient saturation in the highlight areas. The model will automatically increase the contrast threshold of the SIFT algorithm from the default 0.04 to 0.07 to filter out a large number of unstable false edge responses in the overexposed areas. Conversely, when in a cloudy or low-light environment at dusk (such as 2,000 lux), the overall image contrast decreases, and the model will dynamically lower the threshold to 0.02 to ensure that the true texture features in low-contrast areas are not missed. In addition, to prevent abrupt changes in feature point distribution caused by threshold jumps, the system can also introduce temporal smoothing filtering to keep the threshold changes between adjacent frames continuous.

[0072] It should be noted that this step overcomes the shortcomings of traditional methods that use a globally fixed threshold and cannot adapt to the complex lighting changes in outdoor campuses by establishing a nonlinear dynamic mapping between ambient lighting and detection threshold. By utilizing the statistical regularities mined by the implicit correlation model, the feature detector is endowed with "environmental awareness" capabilities, enabling it to automatically adjust its sensitivity according to lighting conditions, thereby maximizing the number of effective features while ensuring the quality of feature points.

[0073] Step S302: In the feature point detection stage, the optimal feature detection threshold is applied to the feature point detection algorithm.

[0074] Feature point detection algorithms refer to computer vision operators used to extract salient and repeatable local features from images, including but not limited to SIFT, SURF, ORB, AKAZE, or deep learning-based algorithms such as SuperPoint and ALIKED. In this embodiment, the algorithm no longer runs as a black-box module, but instead exposes its internal control interface, receiving the optimal feature detection threshold calculated in step S301 as runtime parameters. This allows for dynamic changes in internal selection criteria when processing images acquired at different times, ensuring the physical stability of the output feature points rather than merely the consistency of numerical responses.

[0075] In some implementations, the system explicitly passes Topt into the algorithm's configuration parameter structure when calling the feature point detection function.

[0076] For traditionally manually designed feature operators, the approach is to modify the extreme point screening threshold or non-maximum suppression radius of each layer of the pyramid. For deep learning-based detectors, this manifests as adjusting the cutoff value of the confidence filter layer or adaptively adjusting the downsampling strategy of the feature map.

[0077] For example, when using the SuperPoint network for detection, if Topt indicates a high-noise, high-light environment, the system not only increases the confidence threshold but also automatically enables a mask weighting mechanism for highlight areas, reducing the contribution weight of overexposed pixels to the keypoint heatmap. This parameter injection is completed before the algorithm execution, ensuring that the entire detection process always runs under the optimal configuration of the current environment.

[0078] It should be noted that this step intervenes directly in the core decision-making process of feature extraction, avoiding artifacts or information loss that may be introduced by preprocessing. Furthermore, by closely integrating dynamic thresholds with specific detection algorithms, it ensures that regardless of fluctuations in external lighting, the extracted feature points always represent the true and stable geometric structure of the scene. This also improves the reliability of subsequent feature matching by mitigating image artifacts, imaging distortions, or visual noise caused by lighting conditions.

[0079] Step S303: In the descriptor calculation stage, the environmental state data is encoded as a condition input to constrain the consistency of the generated feature point descriptors under different lighting conditions.

[0080] The descriptor computation stage refers to the process of converting the detected feature point neighborhood image patch into a fixed-length numerical vector, which is used to characterize the local appearance pattern of the feature point in order to perform cross-view matching.

[0081] Encoding environmental state data as conditional input means transforming external environmental parameters such as light intensity and light source orientation angle into high-dimensional vector representations compatible with image features, and injecting them as additional control signals into the descriptor generation network. This breaks the traditional paradigm that descriptors rely solely on pixel intensity, enabling the generated descriptors to explicitly understand the current imaging conditions, thereby aligning the same physical point under different lighting conditions in the feature space.

[0082] In some implementations, the network first extracts initial feature vectors of local image regions through convolutional layers, and simultaneously maps environmental state data into conditional vectors using a multilayer perceptron (MLP). This allows for the concatenation or modulation of the two vectors along the channel dimension. Then, in the AdaIN layer, scaling and shift parameters generated from the conditional vectors are used to perform an affine transformation on the feature distribution. This enables the network to learn the decoupled mapping relationship: "How should the feature distribution of a certain type of texture be transformed under strong side illumination conditions to align with the same type of texture under diffuse reflection conditions?"

[0083] It should be noted that existing methods for robust feature extraction under varying illumination conditions often rely on end-to-end training with large-scale multi-illumination datasets, attempting to implicitly teach the network to ignore illumination changes. However, this often requires massive amounts of data and is difficult to guarantee effectiveness under all extreme conditions. This application, however, injects environmental states as explicit conditional variables into the network, using the AdaIN mechanism to force the network to decouple illumination during the normalization phase. This not only significantly reduces the need for training data but, more importantly, endows the model with explicit physical interpretability (the changes in descriptors are deterministic transformations driven by environmental conditions, rather than uncontrollable black-box fitting). This results in highly robust descriptors that exhibit excellent robustness across time periods and illumination levels, effectively eliminating the phenomenon of increased descriptor distance caused by illumination differences.

[0084] To more clearly illustrate the specific implementation mechanism of conditional adaptive normalization in the above descriptor generation process, a detailed explanation is provided below in conjunction with network structure details. This process essentially involves a structural modification of the standard descriptor generation network, enabling it to possess both environmental awareness and feature calibration capabilities.

[0085] Specifically, the generation of the environment state condition vector is achieved through a lightweight multilayer perceptron network. This network takes ambient light intensity and light source orientation angle as input, and after transformation through several fully connected layers and nonlinear activation functions, outputs a high-dimensional condition vector matching the number of image feature channels. This vector carries a compact semantic representation of the current imaging conditions, essentially providing the network with an "illumination context." Simultaneously, initial feature vectors are extracted from local image regions using conventional convolutional layers. These vectors contain the original texture and geometric information but are mixed with illumination effects. Next, the system concatenates the environment state condition vector with the initial feature vector along the channel dimension or fuses them using a modulation mechanism to obtain a fused feature vector. This ensures that environmental and image information can fully interact at deeper layers of the network.

[0086] Subsequently, the fused feature vector is fed into the conditional adaptive normalization layer of the descriptor generation network. Inside this layer, the environment state condition vector first passes through two parallel fully connected layers to predict the scaling parameters. Translation parameters At this point, the dimensions of these two parameters are consistent with the number of channels in the feature map, representing the degree to which each feature channel should be stretched or compressed and the amount of offset under the current lighting conditions.

[0087] Next, an affine transformation is performed on the fused feature vector using these two parameters, that is, in As input features, and The mean and standard deviation are calculated for instance normalization. This refers to the calibrated features. It can force the feature distribution affected by illumination back to a standard manifold independent of environmental conditions, or map feature distributions under different illumination conditions to the same aligned subspace.

[0088] Finally, based on the feature vector after affine transformation, the final illumination-adaptive feature point descriptor is generated through the fully connected layer or pooling layer in the network head.

[0089] It should be noted that using "environmental state" as a normalized control parameter not only preserves the independence of single-sample processing but also injects precise physical priors, achieving conditional decoupling. This makes the descriptor generation process no longer a blind feature compression but a controlled physical correction process, ensuring the accuracy and stability of cross-illuminance matching.

[0090] Based on the above technical solution, this embodiment constructs a complete illumination-adaptive feature extraction system through a dual mechanism of dynamic threshold control and conditional adaptive descriptor generation. On the one hand, the optimal detection threshold is calculated in real time using a latent association model, ensuring that a sufficient number of reliable feature points can be extracted under various illumination conditions. On the other hand, by explicitly encoding the environmental state and injecting it into the AdaIN layer, deep decoupling and alignment of feature descriptors with illumination changes are achieved. Compared with existing technologies that rely solely on data augmentation or post-processing correction, this embodiment establishes a causal mapping between the environment and features at the algorithm kernel level, which not only improves the success rate of cross-time period image matching but also provides a stable and consistent feature foundation for subsequent high-precision 3D reconstruction.

[0091] Example 4: like Figure 4-5 As shown, this embodiment, based on Embodiment 1, further elaborates on the specific implementation method of adjusting geometric constraint weights during the 3D reconstruction process based on the implicit correlation parameter model set. This embodiment focuses on solving the problem that traditional fixed geometric constraints cannot adapt to changes in outdoor lighting and non-standard geometric shapes on campus. By establishing a dynamic mapping mechanism between lighting conditions, geometric shapes, and constraint strategies, high-fidelity reconstruction of complex structures such as curved walls and irregular decorative surfaces is achieved. The adjustment process of this embodiment mainly includes the following steps: Step S310: Obtain the classification results and geometric parameters of non-standard geometric shapes in the target scene. Non-standard geometric shapes include at least one of curved walls, irregular decorative surfaces, stepped facades, and inclined support columns. Geometric parameters include at least the radius of curvature of the curved area, the height difference of the stepped surfaces, and the tilt angle of the facade.

[0092] Non-standard geometric forms refer to building facade areas that do not conform to standard planar assumptions and have complex spatial topological structures, whose geometric characteristics cannot be accurately described by a single planar equation or regular voxel.

[0093] Geometric parameters are numerical indicators that quantify non-standard forms, serving as necessary intermediate quantities to transform abstract geometric shapes into calculable constraints. It should be understood that while this embodiment lists specific forms such as curved walls and irregularly shaped decorative surfaces, in other embodiments, non-standard geometric forms can also include any architectural structure deviating from a standard plane, such as arched porticoes, spiral staircases, and irregular reliefs, as long as their geometric characteristics can be parameterized.

[0094] In some implementations, deep learning semantic segmentation networks or traditional geometric fitting algorithms are first used to identify and classify non-standard geometric regions from the initially reconstructed sparse point cloud or image sequence. Then, corresponding quantization parameters can be extracted for each type of geometric shape.

[0095] For example, for curved walls, the principal curvature and average radius of curvature are calculated by fitting local quadratic surfaces; For stepped facades, the height difference and depth of each step are determined through planar detection and cluster analysis. For inclined support columns, their tilt angle relative to the direction of gravity is obtained through linear fitting and attitude estimation.

[0096] The parameters described above not only describe the current geometric state but also provide a quantitative basis for determining the initial constraints. Furthermore, to improve the robustness of parameter extraction, the system can also combine multi-frame observation data for temporal smoothing or joint optimization to remove outliers caused by noise or occlusion.

[0097] It should be noted that this step transforms qualitative geometric shapes into quantitative geometric parameters, enabling the reconstruction algorithm to "perceive" the specific characteristics of the geometric shape through refined parameter representation. This allows the algorithm to match the most suitable constraint strategy for different shapes, avoiding the accuracy loss caused by "one-size-fits-all" processing.

[0098] Step S320: Based on the classification results and geometric parameters of non-standard geometric shapes, determine the geometric type of the area to be reconstructed in the target scene and the corresponding initial constraint smoothness parameters; based on the solar altitude angle and wall orientation at the current image acquisition time, call the correlation model between the light source direction and the scene shadow distribution in the implicit correlation parameter model set to calculate the shadow distribution characteristics of the area to be reconstructed under the current lighting conditions. The shadow distribution characteristics include at least the shadow coverage and the shadow boundary position.

[0099] The initial constraint smoothness parameter refers to the surface fitting regularization coefficient set based solely on the geometric shape itself when the influence of illumination is not considered. It is used to balance the weights between the data fitting term and the surface smoothness term.

[0100] Shadow distribution features refer to the distribution of dark areas and their boundary information formed on the surface of a target scene due to self-occlusion or mutual occlusion under a specific light source direction. It is a key bridge connecting ambient lighting and geometric structure. Its shadow distribution is not a pixel-level mask obtained by segmentation from a single image, but a theoretical distribution predicted based on three-dimensional geometry and the physical laws of lighting. Therefore, it is not affected by imaging quality issues such as overexposure, underexposure, or texture loss.

[0101] In some implementations, the initial smoothness parameters for each region are determined by querying a preset geometry-constraint mapping table based on the geometric parameters obtained in step S310. For example, for sharp surfaces with small radii of curvature, the initial smoothness is set to a lower value to preserve details; for large-area smooth surfaces, the initial smoothness is set to a higher value to suppress noise.

[0102] The system acquires the solar altitude angle and wall orientation at the current acquisition time, using these as input variables, and calls the correlation model between the light source direction and scene shadow distribution constructed in Example 2. This model outputs the expected shadow coverage under the current lighting conditions and the projection position of the shadow boundary in three-dimensional space. Specifically, if the model predicts that a curved surface area is under side lighting and has significant self-occlusion, the output shadow boundary position will be marked as a "high-confidence structure line"; if the prediction is uniform diffuse lighting, the shadow coverage is zero, and no effective structure line is output, ensuring the accuracy and stability of the shadow information.

[0103] It should be noted that in outdoor campus scenes, shadow boundaries often correspond to real building outlines or decorative lines, making them a more reliable source of geometric constraints than texture features. However, the presence or absence of shadows is highly dependent on the lighting angle, and traditional methods struggle to dynamically utilize this information. This step introduces shadow distribution features, successfully transforming ambient lighting information into a structured prior usable for geometric reconstruction. This allows the system to fully utilize shadows as strong constraints when they are present, and automatically switch to other constraint modes when there are no shadows, achieving deep coupling between environmental information and geometric reconstruction.

[0104] Step S330: Based on the shadow distribution characteristics, adjust the initial constraint smoothness parameters to generate optimized constraint smoothness parameters that match the current lighting conditions and non-standard geometric shapes; identify the curved surface regions in the target scene and extract the surface curvature radius of the curved surface regions; adjust the smoothness parameters of the surface fitting according to the surface curvature radius and the current lighting conditions.

[0105] The optimized constraint smoothness parameter refers to the final surface fitting control parameter dynamically generated after integrating prior geometric features and real-time lighting conditions. This parameter is no longer a static constant, but a dynamic variable that changes in real time with environmental conditions and geometric features. Moreover, the adjustment process is essentially a multi-factor decision-making process, aiming to find the optimal balance between geometric fidelity and noise suppression capability under the current operating conditions.

[0106] In some implementations, the system uses a combination of conditional branching logic and continuous functions for adjustment. For curved surfaces, the radius of curvature is first determined. With preset curvature threshold The relationship is considered, and a differentiated strategy is implemented based on the current lighting conditions (side lighting or top lighting). When the conditions are met... Furthermore, under side lighting conditions, this indicates that the area is a sharp, detailed surface with enhanced edge visibility due to shadows. In this case, the system reduces the smoothness parameter. To preserve high-frequency details while increasing the weight of line segment constraints The predicted shadow boundary is used as a strong guiding line to constrain the surface fitting direction. Conversely, when the following conditions are met... Furthermore, under positive top lighting conditions, it indicates that the region is a gently curved surface and the disappearance of shadows leads to blurred edges. In this case, the system increases the smoothness parameter. To suppress surface undulations caused by noise and increase the normal constraint weight of the surface patch. The reconstruction quality is ensured by relying on the continuity of surface normals. The above adjustment logic can be formally represented by the following piecewise function:

[0107] in, The optimized smoothness parameters, The initial smoothness parameters determined in step S320, The attenuation coefficient is less than 1 (e.g., 0.5). An enhancement factor greater than 1 (e.g., 2.0). and These represent the Boolean decision results for side lighting and top lighting conditions, respectively. For the logical AND operator. Additionally, there are line segment constraint weights. With normal constraint weight A similar dynamic adjustment function or lookup table mechanism can also be used to determine this.

[0108] It should be noted that by establishing a ternary dynamic mapping of lighting, geometry, and constraints, the constraint strategy can intelligently switch according to environmental conditions: under side lighting, it "trusts" the edge information provided by shadows, and under top lighting, it "trusts" the smoothness information provided by surface continuity. This not only improves the reconstruction accuracy of non-standard geometric shapes, but also endows the system with the ability to work stably in a variable campus environment, effectively overcoming the systematic bias of fixed constraint methods.

[0109] Step S340: Based on the optimized constraint smoothness parameters, perform surface fitting and surface optimization on the point cloud data of the region to be reconstructed to generate a reconstructed surface with non-standard geometric shape; based on the dynamically adjusted smoothness parameters, the weight of line segment constraints and the weight of patch normal constraints, perform point cloud optimization and surface fitting on the surface region.

[0110] Surface fitting and surface optimization refer to the process of recovering a continuous, smooth, and geometrically prior 3D surface model from discrete, noisy point cloud data using mathematical optimization algorithms. This process integrates the optimization constraint parameters dynamically generated in step S330. Specifically, these parameters are directly embedded into the energy minimization equation of surface reconstruction, serving as coefficients of the regularization term to guide the optimization direction. It should be understood that although this embodiment uses point cloud surface fitting as an example, in other embodiments, this step is also applicable to other 3D representations such as mesh optimization and implicit surface reconstruction (e.g., NeRF, SDF), as long as their optimization objective function includes adjustable geometric constraint terms.

[0111] In some implementations, the system constructs an energy function of the following form for solving:

[0112] in, The surface to be optimized; This is a data fitting term used to measure how closely the surface approximates the original point cloud; For smoothing regularization terms; This is a line segment constraint term that measures the alignment between the surface edge and the predicted shadow boundary; This is the normal consistency term, which measures the continuity of the normals of adjacent facets.

[0113] In scenes with sharp curved surfaces illuminated by side lighting, due to Smaller and The surface is relatively large, so the optimization process will prioritize ensuring that the curved edges fit the shadow boundaries, even at the cost of sacrificing local smoothness, so as to accurately restore details such as window frames and moldings; In scenes with gently curved surfaces illuminated from above, due to and The larger the surface, the more the optimization process will force the surface to remain smooth and continuous, effectively filtering out false fluctuations caused by uneven lighting or sensor noise.

[0114] By iteratively solving the energy function, a high-precision reconstructed surface that conforms to both geometric priors and current lighting conditions is finally generated.

[0115] By explicitly embedding dynamic parameters into the optimization equations, it is ensured that adjustments to the constraint strategies truly impact the reconstruction results, rather than merely remaining at the parameter level. This closed-loop design of perception, decision-making, and execution enables the system to consistently output optimal reconstruction quality when faced with the complex geometry of non-standard campus walls and varying lighting environments.

[0116] Based on the above technical solution, this embodiment constructs an adaptive reconstruction system driven by both illumination and geometry through a complete chain of geometric parameterization, shadow distribution prediction, dynamic constraint adjustment, and adaptive surface fitting. This system overcomes the limitations of traditional fixed constraint methods, utilizing the physical laws of edge enhancement under side lighting and surface continuity dominance under top lighting to achieve intelligent switching of constraint strategies according to environmental conditions. This not only effectively solves the reconstruction distortion problem of non-standard geometric shapes under complex lighting but also provides a high-precision, highly robust 3D data foundation for applications such as campus digital twins and digital preservation of ancient buildings.

[0117] Example 5: like Figure 6-7As shown, this embodiment, based on Embodiment 1, further elaborates on the spatiotemporal dynamic modeling and compensation reconstruction mechanism for dynamic occlusions in the target scene. Climbing plants, temporary construction barriers, seasonal vegetation, and other occlusions commonly found in campus environments often result in holes or broken connections in the model when using traditional static masking removal methods. This embodiment effectively solves the problem of missing 3D model integrity caused by dynamic occlusion by upgrading the occlusion processing from single-frame static detection to cross-time-period spatiotemporal complementary reconstruction. The processing flow of this embodiment mainly includes the following steps: Step S500: Create spatiotemporal labels for occlusions in the target scene, including the time of occurrence and the degree of impact.

[0118] Spatiotemporal tags are used to quantify the existence of occluders in the four-dimensional spatiotemporal dimension and their impact on image quality. These tags not only record the semantic category of the occluder but also include its dynamic evolution over time. Specifically, the spatiotemporal tag data structure includes at least three core fields: occluder type (e.g., evergreen vines, deciduous trees, temporary scaffolding, air conditioner outdoor units, etc.), occurrence time (accurate to season, month, or even specific date range, such as "June-September" or "14:00-16:00 daily"), and impact level (divided into three levels: mild, moderate, and severe, corresponding to partially blurred texture, partially obscured structure, and completely invisible, respectively).

[0119] In some implementations, a deep learning object detection network (such as Mask R-CNN or YOLO series) is first used to identify the acquired image sequence frame by frame, extracting the initial bounding boxes and category confidence scores of the occluders. Then, the detection results of the same spatial location at different timestamps can be temporally aligned and clustered, and the frequency of occurrence and coverage area changes of various occluders in different seasons and time periods can be statistically analyzed.

[0120] Based on the aforementioned statistical data and the correlation model mined in Example 2, the life cycle patterns of specific occlusions are inferred, and standardized spatiotemporal labels are generated accordingly. For example, if the model shows that a certain wall area is covered by green vegetation with a coverage rate exceeding 80% in images from July to August each year, but the coverage rate is less than 10% in images from November to March of the following year, the system automatically generates labels for that area: {Type: Deciduous Vine, Time Period: July-August, Severity: Severe} and {Type: Deciduous Vine, Time Period: November-March, Severity: Mild}. This data-driven label generation mechanism enables the system to accurately grasp the spatiotemporal distribution patterns of occlusions, providing a reliable basis for subsequent adaptive processing.

[0121] It is worth noting that explicitly encoding the dynamic changes of occlusions into structured knowledge enables the reconstruction system to "predict" when, where, and to what extent occlusion will occur, thus transforming passive removal into proactive management. This not only improves the accuracy of occlusion handling but also lays a data foundation for subsequent compensation reconstruction using complementary time-period data.

[0122] Step S510: In the feature matching stage, based on the spatiotemporal labels, reduce the matching weight of feature points in the occluded area.

[0123] In this context, the matching weight refers to the coefficient used to measure the contribution of a feature point to camera pose estimation during the calculation of feature point descriptor similarity.

[0124] Reducing the matching weight of feature points in occluded areas refers to dynamically suppressing the influence of feature points located in known occluded areas or severely affected by occlusion in the feature matching algorithm based on the spatiotemporal labels generated in step S500, so as to prevent incorrect matching caused by unstable textures or repetitive patterns at the edges of occluded objects from polluting the global optimization results.

[0125] It should be understood that reducing weights is a continuous and gradual process, rather than a simple binarization removal. For lightly occluded regions, the weight may only be reduced to 0.7 to retain some effective information; for heavily occluded regions, the weight may be suppressed to 0.1 or even close to zero, so that it hardly participates in the decision-making process in RANSAC or Bundle Adjustment optimization.

[0126] In some implementations, the spatiotemporal label corresponding to the current image acquisition time is queried in real time when constructing the feature matching map. For each detected feature point, it is first determined whether its pixel coordinates fall within the effective occlusion mask range of the current time period. If it falls into a heavily occluded area, the matching score of the feature point is directly multiplied by a decay factor. If the area falls into a slightly shaded area or a transition zone between shading boundaries, the attenuation factor is calculated by linear interpolation based on the degree of shading.

[0127] In addition, considering that the edges of occluded objects often contain mixed pixels (i.e., a single pixel contains information about both the wall and the occluded object), the system will also combine gradient direction consistency checks to impose additional penalties on feature points whose gradient directions are parallel to the occlusion boundary, because these points are highly likely to be the outline of the occluded object rather than the wall structure.

[0128] It should be noted that by injecting spatiotemporal prior knowledge into the core decision-making process of feature matching, the first line of defense against dynamic interference is constructed. This allows for "preventive intervention" using spatiotemporal labels before matching occurs, reducing the number of erroneous matching candidates at the source and improving the convergence speed and stability of the algorithm in complex occlusion environments.

[0129] Step S520: In the multi-view fusion stage, complementary image information collected at different time periods is used to compensate and reconstruct the occluded area.

[0130] Compensation reconstruction refers to the process of utilizing the complementary visibility of the target scene at different points in time to transfer and fill in the missing parts of the model at the current time with information from areas that are occluded at one time but clearly visible at another, thereby restoring complete geometry and texture. This process is based on cross-temporal and spatial information fusion of real observation data.

[0131] It should be understood that the “complementary image information” here includes not only RGB texture colors, but also geometric and attribute information such as depth values, normal vectors, and semantic labels. Moreover, this information must undergo strict spatiotemporal alignment and illumination normalization to ensure that the fused model is geometrically continuous and visually consistent.

[0132] In some implementations, the system iterates through the image sequences of all acquisition time periods, combining the spatiotemporal labels and occlusion mask generated in step S500 to record the visibility status (visible, occluded, partially visible) of each facet or voxel unit on the 3D model surface in each frame of the image. When a target surface region is found to be marked as "occluded" in the first image acquired at time T1, the system immediately queries the visibility index of that region to find a second image acquired at time T2 that is marked as "visible". Once a complementary image that meets the criteria is found, the system initiates the data migration process.

[0133] For example, for a brick wall completely covered by dense ivy in summer, after the system detects a heavy occlusion label at the corresponding location in a summer image, it will automatically retrieve images taken in winter or early spring to extract clear textures and accurate geometric point clouds of the brick wall area as a compensation data source. This avoids blind full-database search and significantly improves the efficiency of compensation reconstruction.

[0134] It should be noted that in a campus setting, while the spatiotemporal changes of obstructions pose challenges to single data collection, they also provide opportunities for observation from multiple angles and in multiple states. This application utilizes this dynamism to compensate for the shortcomings of static observations by establishing complementary correlations across time periods. Compared to generative AI filling in potentially spurious details, the compensation data used in this step originates from direct observations of the real physical world, ensuring the authenticity and credibility of the reconstruction results.

[0135] Step S530: Obtain image sequences from various perspectives and time periods in the target scene, as well as the acquisition timestamps and spatiotemporal labels corresponding to each image; based on the spatiotemporal labels, mark the occlusion areas of each image in the image sequence to generate an occlusion mask map; using the 3D model of the target scene as a spatial reference, project the images acquired at each time period onto the corresponding surface areas of the 3D model, and determine the visibility status of each surface area in each image based on the occlusion mask map; for target surface areas marked as occluded in the first image acquired at the first time and marked as visible in the second image acquired at the second time, extract the texture information and geometric constraint information of the corresponding areas in the second image; map the texture information and geometric constraint information to the corresponding positions of the target surface areas in the 3D model according to the projection mapping relationship, generate compensation texture and compensation geometric data, and perform surface filling and smoothing optimization on the target surface areas accordingly to obtain the 3D model after occlusion compensation.

[0136] The occlusion mask is a two-dimensional binary or probability map with the same size as the original image, used to accurately identify whether each pixel belongs to an occlusion or a background wall.

[0137] The determination of visibility state is a judgment process that combines three-dimensional geometric projection and two-dimensional mask query, which is used to solve the problem of inconsistent occlusion judgment caused by differences in viewpoint.

[0138] Extraction and mapping of texture and geometric constraint information are the core execution steps of compensation reconstruction, involving cross-view texture reprojection and geometric alignment. It should be understood that its smoothing optimization is not a simple blur filtering, but rather eliminates seam traces caused by lighting differences or registration errors while maintaining the geometric continuity and texture consistency between the compensation area and the original area.

[0139] In some implementations, when generating the occlusion mask map, the system not only relies on the output of the semantic segmentation network, but also combines depth discontinuity detection and motion parallax analysis to perform fine processing or sub-pixel-level fine adjustment on the mask edges to ensure that the occlusion boundary accurately fits the actual object contour.

[0140] When determining the visibility state, the surface of the 3D model is discretized into micro-facets. The center point of each micro-facet is projected back to the coordinate system of each frame image, and it is checked whether the projection point falls within the non-occluded area of ​​the occlusion mask. At the same time, it is verified whether the angle between the projection ray and the normal of the facet is less than a preset threshold (such as 75 degrees) to avoid low-quality observations at grazing angles being misjudged as valid visibility.

[0141] During the compensation mapping process, inverse texture mapping technology is employed to accurately resample pixels from the second image onto the surface of the 3D model based on camera intrinsic and extrinsic parameters. Simultaneously, to eliminate color differences caused by variations in illumination at different times, the system invokes the illumination adaptive normalization mechanism described in Example 3 before mapping, or utilizes a histogram matching algorithm to perform tone shifting on the compensation texture, ensuring consistency in brightness and color temperature with existing surrounding textures.

[0142] In the surface filling and smoothing optimization stage, a local energy minimization problem can be constructed. Geometric continuity constraints (such as Laplacian smoothing) and texture gradient consistency constraints are applied at the boundary between the compensation region and the original region. Through iterative solution, the geometric transition at the seam is made natural and the texture is seamlessly integrated.

[0143] For example, when using the clear brick wall texture of winter to compensate for areas obscured by vines in summer, the system automatically adjusts the white balance of the winter texture to match the warm tones of the summer environment, and performs Poisson blending on the splicing edges, ultimately outputting a complete wall model that retains the geometric details of winter while conforming to the overall lighting atmosphere of summer.

[0144] It should be noted that by establishing a complete closed loop of spatiotemporal labeling, visibility indexing, inverse texture mapping, and illumination consistency correction, the compensation reconstruction is no longer arbitrary image pasting, but a precise synthesis process guided by physical constraints and prior knowledge. In particular, the introduction of a dual verification mechanism for visibility states (mask query + normal verification) and the illumination consistency correction stage effectively solves the problems of geometric misalignment and visual fragmentation in cross-time period data fusion, ensuring that the final output 3D model meets industrial-grade standards in both geometric accuracy and visual realism.

[0145] Based on the above technical solution, this embodiment constructs an adaptive reconstruction system for dynamic occlusion environments on campuses through end-to-end processing including spatiotemporal label modeling, matching weight suppression, complementary visibility retrieval, and precise compensation fusion. This system overcomes the limitations of traditional static occlusion processing, transforming temporal information into a key resource for solving spatial missing information problems, achieving a shift from passively tolerating occlusion to actively utilizing temporal complementarity. This not only improves the integrity and usability of 3D models in complex campus environments but also provides seamless, high-fidelity spatial data support for subsequent digital twin applications.

[0146] Example 6: like Figure 8As shown, this embodiment provides a 3D reconstruction and parameter calibration system for non-standard campus walls. This system serves as the hardware carrier and functional integration platform for the methods described in the aforementioned embodiments. The system architecture of this embodiment not only automates the entire 3D reconstruction process but also internalizes the adaptive illumination calibration and correction function as a native capability of the system. Deep collaboration between modules solves the problem of poor environmental adaptability caused by the separation of traditional calibration and reconstruction. Specifically, the system includes a data acquisition module, a correlation analysis module, a dynamic parameter adjustment module, and a 3D reconstruction module.

[0147] The data acquisition module is the system's perception entry point. Its hardware components include, but are not limited to, a multispectral camera array, an UAV oblique photography payload, a ground laser scanner, and a synchronously integrated micro-weather station or light sensor group, used to acquire image data of the target scene and corresponding environmental status data.

[0148] In a preferred implementation, the data acquisition module integrates a high-precision time synchronization unit (such as a PTP protocol clock or a GPS timing module) to ensure that the acquisition time of each frame of image data is aligned with environmental state data (light intensity, solar altitude angle, weather type, etc.) at the microsecond level. This hardware-level time synchronization mechanism provides a noise-free raw data foundation for the subsequent correlation analysis module to explore the causal relationship between environmental variables and image features.

[0149] It should be understood that although this embodiment lists the specific sensor types mentioned above, in other embodiments, the data acquisition module may also acquire environmental data from a third-party meteorological API or campus IoT platform through a network interface, as long as the time alignment accuracy of the data meets the input requirements of the implicit correlation model.

[0150] The correlation analysis module is the "cognitive center" of the system. It stores a set of parameterized models that have been pre-constructed and verified by the method in Example 2. These models are used to analyze the set of implicit correlation parameter models between environmental state data and image features. The implicit correlation relationship includes at least one of the following: a nonlinear mapping model between ambient light intensity and global image brightness, a statistical correlation model between ambient light intensity and feature point detection repetition rate, and a correlation model between light source direction and scene shadow distribution. These models are used to characterize the influence of different environmental states on the performance of image features.

[0151] During runtime, the module receives real-time environmental state vectors from the data acquisition module and quickly outputs the expected prior parameters of image features under the current environmental conditions through methods such as table lookup, function fitting, or neural network inference. For example, when the input is strong midday light, the module outputs a high feature detection threshold suggestion value and a low shadow coverage prediction value; when the input is sidelight in the morning, it outputs a low threshold suggestion value and a high-confidence line segment constraint region mask.

[0152] It should be noted that the output of the correlation analysis module is not merely static data query results, but also dynamic control signals that drive downstream modules to make adaptive adjustments. This allows the system to detect potential quality risks and implement proactive countermeasures before image processing.

[0153] The parameter dynamic adjustment module acts as the system's "execution controller," responsible for transforming the abstract prior knowledge output by the association analysis module into specific configuration parameters that the reconstruction algorithm can recognize. Internally, it contains two sub-units: a feature parameter mapper and a constraint weight scheduler. Based on the optimal threshold, normalization coefficients, and other parameters output by the model, it dynamically modifies the runtime configuration of the feature extraction algorithm. The constraint weight scheduler, on the other hand, generates the regularization coefficient matrix in the surface fitting energy equation in real time based on geometric morphology classification and shadow distribution prediction.

[0154] In some implementations, the parameter dynamic adjustment module can be implemented using an FPGA or a dedicated ASIC chip to meet the real-time requirements of parameter adjustment in 3D reconstruction; alternatively, it can be implemented as a software service running on a general-purpose processor, interacting with the reconstruction engine through shared memory or message queues. Regardless of the hardware form used, the core function of this module is to establish a low-latency feedback loop from "environmental perception" to "algorithm control," ensuring that the reconstruction process always operates within the optimal parameter space under the current environmental conditions.

[0155] The 3D reconstruction module is the system's "production engine," integrating standard 3D vision algorithm libraries such as SfM sparse reconstruction, MVS dense matching, mesh generation, and texture mapping. It is used to perform 3D reconstruction on image data based on at least one of the adjusted feature extraction parameters and geometric constraint weights to obtain a 3D model of the target scene.

[0156] Unlike the previous embodiments, the 3D reconstruction module in this system provides an open parameter injection interface, allowing the parameter dynamic adjustment module to intervene in its internal algorithm behavior at runtime. For example, in the feature matching stage, the reconstruction module directly reads the dynamically adjusted threshold and descriptor generation network weights; in the surface optimization stage, it directly loads the dynamically generated smoothness parameters and line segment constraint weights.

[0157] Building upon this, the system in this embodiment also includes an illumination adaptive calibration and correction module. This module is not an additional component independent of the aforementioned architecture, but rather a system-level built-in function implemented collaboratively by the correlation analysis module and the parameter dynamic adjustment module. The illumination adaptive calibration and correction module is used to synchronously record environmental state data in response to the camera calibration start signal; evaluate the reliability of corner detection on the calibration board based on the implicit correlation parameter model set; and, in response to the calibration result optimization start signal, remove or reduce the weight of low-reliability calibration images, outputting camera calibration parameters with confidence assessment.

[0158] Specifically, when the system receives the camera calibration start signal, the data acquisition module automatically switches to calibration mode. While acquiring the calibration board image sequence, it simultaneously records the ambient light intensity, light source direction, and weather type for each frame at high frequency. Subsequently, the correlation analysis module calls a preset statistical correlation model between ambient light intensity and feature point detection repetition rate to perform real-time quality assessment on each calibration image.

[0159] Based on historical calibration data mining, the statistical distribution of corner sub-pixel localization errors under specific lighting conditions is quantified. For example, when the illumination intensity of a frame exceeds 80,000 lux and there is local overexposure, the model outputs a corner detection confidence score of 0.3 (out of 1.0) for that frame; while for images with uniform illumination and moderate intensity, the confidence score can reach above 0.95. This evaluation process fully reuses the core correlation analysis capabilities of the system, without the need to deploy an additional independent calibration quality evaluation algorithm, demonstrating the compactness of the architecture and the reusability of the functions.

[0160] Furthermore, the parameter dynamic adjustment module receives the confidence score from the correlation analysis module and converts it into weighting factors in the calibration optimization equation. During Bundle Adjustment or camera intrinsic parameter optimization, the system constructs the following weighted objective function:

[0161] in, The sum of reprojection errors is given by N, where N is the total number of calibration images. Let be the coordinates of the corner point detected in the i-th image. Let K be the camera projection function, and K be the intrinsic parameter matrix to be optimized. X is the extrinsic parameter, and X represents the coordinates of a 3D point on the calibration plate. The key lies in the weights. Determination: The system directly assigns the credibility score output by the association analysis module to... Or set a threshold When the credibility is lower than season =0 (i.e., discard the frame), otherwise This is equivalent to the confidence score itself. In this way, the negative impact of low-quality calibration images on the calibration results is automatically suppressed or even completely eliminated, while the contribution of high-quality images is preserved. The final output camera calibration parameters not only include the intrinsic parameter matrix and distortion coefficients, but also a comprehensive confidence index, which reflects the overall quality level of the dataset used in this calibration, providing a reliable accuracy expectation for the subsequent 3D reconstruction module.

[0162] It is worth noting that by leveraging the system's existing implicit correlation analysis capabilities, low-quality data under adverse environmental conditions can be automatically identified and filtered, freeing the calibration operation from stringent environmental requirements. More importantly, this collaborative mechanism ensures that calibration and reconstruction parameters are based on the same environmental understanding system, avoiding systematic biases introduced by inconsistencies in the environmental understanding between calibration and reconstruction. For example, if the correlation analysis module determines that the current environment is one of strong side lighting, it will not only reduce the weight of the calibration image but also automatically enhance line segment constraints in subsequent reconstruction to compensate for feature uncertainties caused by uneven illumination, thereby ensuring logical consistency and performance stability throughout the entire process from calibration to reconstruction.

[0163] Example 7: This embodiment provides a comprehensive application scenario for a method of 3D reconstruction and parameter calibration of non-standard campus walls. This embodiment aims to systematically connect and verify the dispersed technical features of the aforementioned embodiments through a real campus building case involving complex geometry, dynamic vegetation shading, and all-weather lighting changes, to intuitively demonstrate the synergistic effect and superior performance of the technical solution in solving practical engineering problems.

[0164] Step S700: Obtain multi-time-period, multi-modal data of the target scene and initialize the implicit correlation parameter model set.

[0165] In this embodiment, the target scene is specifically set as the east facade of an old campus library, which has typical high complexity features. Geometrically, it includes an arc-shaped curved wall with a radius of about 3.5 meters and irregular brick decorative lines; environmentally, it is covered with seasonal deciduous climbing plants (Witch ivy), and its due east orientation causes drastic changes in lighting conditions throughout the day.

[0166] Multi-time-period multimodal data refers to the collection of image sequences and environmental sensor data collected in this scenario across the four seasons of spring, summer, autumn, and winter, as well as the three typical time periods of morning, noon, and dusk each day.

[0167] Initializing the set of implicit correlation parameter models refers to loading the causal correlation model library that has been pre-trained by the method in Example 2 and optimized for the microclimate characteristics of the campus area.

[0168] Specifically, the team used drones and ground cameras to collect high-resolution images of the wall 12 times over a year. Each collection simultaneously recorded environmental parameters such as illuminance, solar altitude angle, temperature, and humidity, accumulating 3,600 sets of aligned samples. Based on this historical data, the system has constructed a parameter set including 12 core implicit correlation models such as "illuminance-feature response," "solar angle-shadow distribution," and "season-vegetation coverage."

[0169] When this reconstruction task is initiated, the system automatically loads the model set as a prior knowledge base, enabling subsequent processing to perceive and adapt to the environmental characteristics of this specific scenario in real time.

[0170] Step S710: Under morning side lighting conditions, dynamically enhance line segment constraints based on shadow distribution characteristics to reconstruct surface details.

[0171] Among them, morning side lighting conditions refer to the period when the sun's altitude angle is below 30 degrees and the light shines on the wall at a small incident angle. At this time, the uneven structure of the wall will produce a significant elongated shadow.

[0172] Dynamically enhanced line segment constraints refer to invoking the geometric constraint adjustment mechanism of Example 4 to increase the weight coefficient of the line segment guiding term in the surface fitting energy function based on the real-time predicted shadow boundary position.

[0173] Specifically, when the system acquires images at 8:00 AM, the environmental sensor measures the solar altitude angle to be 22 degrees. The correlation analysis module immediately calls the correlation model between the light source direction and the scene shadow distribution to accurately predict the theoretical shadow boundaries of the window frame, brick seam, and curved surface transition under the current lighting.

[0174] Given that the shadows enhance the contrast of the structural edges, the parameter dynamic adjustment module automatically increases the line segment constraint weight from the default value of 0.3 to 0.8, while reducing the smoothness parameter by 40%. With this configuration, the 3D reconstruction module prioritizes using high-confidence shadow boundary lines as geometric guides. Even with weak textures in curved areas, it can accurately recover the arc shape with a curvature radius of 3.5 meters and the sharp edges of the brickwork lines by relying on strong line segment constraints, effectively avoiding the detail erasure phenomenon caused by excessive smoothing in traditional methods.

[0175] Step S720: Under noon overhead lighting conditions, switch to illumination adaptive feature extraction and normal constraint mode to suppress noise.

[0176] Among them, the midday overhead light condition refers to the period when the sun's altitude angle is greater than 60 degrees and the light shines almost perpendicularly on the wall. At this time, the projection disappears and local overexposure is likely to occur.

[0177] Switching to the illumination-adaptive feature extraction and normal constraint mode means, according to the strategies of Examples 3 and 4, enabling the Conditional Adaptive Normalization (AdaIN) descriptor generation network and shifting the geometric constraint focus from line segments to the normal continuity of the surface.

[0178] Specifically, when the system acquired images at 12:00 noon, the light intensity reached as high as 85,000 lux, and the correlation analysis module determined that the current condition was "strong light - weak texture". The feature extraction module then invoked the correlation model between light intensity and feature threshold, raising the detection threshold to 0.07 to filter overexposed noise, and injected a strong light conditional vector through the AdaIN layer to force calibration of the feature distribution to match the feature space of the morning data. At the same time, because the disappearance of shadows caused the line segment constraints to fail, the parameter dynamic adjustment module automatically increased the normal constraint weight to 0.9 and increased the smoothness parameter.

[0179] In this mode, the reconstruction algorithm no longer relies on unreliable edge information, but instead relies on the consistency of surface normals to maintain surface smoothness, effectively suppressing false surface undulations and particle noise caused by strong light diffuse reflection, and ensuring seamless geometric connection between noon data and morning data.

[0180] Step S730: Establish a cross-seasonal shading complementarity mechanism based on spatiotemporal labels, and use winter data to compensate for summer vegetation shading areas.

[0181] Among them, the cross-seasonal shading complementarity mechanism refers to the application of the spatiotemporal dynamic shading modeling and compensation reconstruction method in Example 5, which utilizes the temporal pattern of vegetation growth cycle to establish an information migration channel between data from different seasons.

[0182] Specifically, in images acquired in summer (July), the semantic segmentation network identified that 60% of the lower part of the wall was covered by dense ivy. The system automatically generated spatiotemporal labels {Type: Deciduous Ivy, Time Period: June-September, Severity: Severe}. During the feature matching stage, the matching weights of feature points in this area were automatically suppressed to 0.05 to prevent incorrect matches from contaminating the pose estimation.

[0183] After entering the multi-view fusion stage, the system queries the global visibility index and finds that the same area is marked as {Type: Deciduous Vines, Time Period: November-February, Degree: Mild / Withered} in images acquired in winter (January), and the visibility is above 95%. Therefore, the system automatically extracts the clear brick wall texture and accurate point cloud of this area from the winter image, corrects it by illumination normalization, and accurately fills the occluded holes in the summer model through inverse texture mapping.

[0184] Step S740: Integrate the adaptive reconstruction results from multiple time periods to output a high-precision 3D model with no color difference and no holes.

[0185] Among them, the fusion of multi-time adaptive reconstruction results refers to the process of globally optimizing and uniformly expressing the sparse point cloud, dense point cloud and texture information after the above-mentioned differentiated processing.

[0186] A high-precision 3D model with no color difference and no holes is the quality standard for the final deliverable of the technical solution in this application.

[0187] Specifically, after completing the adaptive reconstruction and occlusion compensation of all time period data, the system performs global Bundle Adjustment optimization. At this time, all feature points involved in the optimization have achieved illumination decoupling through the AdaIN mechanism in Example 3. The descriptor distance metric is no longer affected by the illumination difference at the time of acquisition, thereby ensuring the geometric consistency of matching points across time periods.

[0188] During the texture mapping stage, the system uses the "lighting-brightness nonlinear mapping" in the implicit correlation model to physically align the textures for each time period, eliminating the stitching marks between the cool tones of the morning and the warm tones of midday. The final output is a 3D model.

[0189] Based on the above technical solutions, this embodiment fully verifies the actual effectiveness of the technical system of environmental perception, parameter adaptation, and spatiotemporal complementarity by executing the core methods of the aforementioned embodiments in a real campus non-standard wall scene in a sequential manner according to time and season. This application scenario not only intuitively addresses the four major pain points raised in the background technology, such as temporal illumination changes, color difference, occlusion breaks, and non-standard deviations.

[0190] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for three-dimensional reconstruction and parameter calibration of a non-standard wall surface in a campus, characterized in that, include: Acquire image data and corresponding environmental state data of the target scene; The set of implicit correlation parameter models between the environmental state data and image features is analyzed. The set of implicit correlation parameter models includes at least one of the following: a nonlinear mapping model between ambient light intensity and global image brightness, a statistical correlation model between ambient light intensity and feature point detection repetition rate, and a correlation model between light source direction and scene shadow distribution. These models are used to characterize the influence of different environmental states on image feature performance. Based on the set of implicit correlation parameter models, at least one of the feature extraction parameters and geometric constraint weights in the 3D reconstruction process is adjusted. Based on at least one of the adjusted feature extraction parameters and geometric constraint weights, three-dimensional reconstruction is performed on the image data to obtain a three-dimensional model of the target scene.

2. The method according to claim 1, wherein, The construction process of the implicit correlation parameter model set includes: Acquire historically collected multidimensional time-series data, including historical environmental state data aligned to timestamps and historical image feature data; The multidimensional time series data is reconstructed in phase space, and the reconstructed phase space trajectory is generated by delay coordinate embedding method; A convergent cross-mapping algorithm is used to test the causal relationship of the phase space trajectory. By calculating the cross-mapping prediction error, the causal driving direction and strength between the historical environmental state data and the historical image feature data are determined. Dynamic factor analysis is used to extract potential common driving factors from multidimensional variables with the aforementioned causal driving direction and intensity, in order to decouple the coupling effects under complex environmental conditions; Based on the causal driving direction and strength and the common driving factors, the mathematical expressions of the nonlinear mapping model, the statistical association model and the association model are quantified and generated, and a confidence score is assigned to each model to form the parameterized model set. The assignment of the confidence score is based on statistical significance and causal test strength.

3. The method of claim 1, wherein, The process of adjusting the feature extraction parameters in the 3D reconstruction process based on the implicit correlation parameter model set includes: Based on the ambient light intensity at the current image acquisition time, and based on the implicit correlation, the optimal feature detection threshold is calculated; In the feature point detection stage, the optimal feature detection threshold is applied to the feature point detection algorithm; During the descriptor computation phase, the environmental state data is encoded as a conditional input to constrain the consistency of the generated feature point descriptors under different lighting conditions.

4. The method according to claim 3, wherein, The process of encoding the environmental state data as conditional input during the descriptor calculation stage to constrain the consistency of the generated feature point descriptors under different lighting conditions includes: The ambient light intensity and the light source direction angle are extracted from the environmental state data, and the ambient light intensity and the light source direction angle are mapped into a high-dimensional environmental state condition vector through a multilayer perceptron network. In the descriptor generation network, initial feature vectors of local image regions are extracted through convolutional layers; The environmental state condition vector and the initial feature vector are concatenated along the channel dimension to obtain the fused feature vector; The fused feature vector is input into the conditional adaptive normalization layer of the descriptor generation network. Using the environmental state condition vector, scaling and translation parameters are calculated through a fully connected layer to perform an affine transformation on the fused feature vector to calibrate the feature distribution. Based on the feature vector after affine transformation, an illumination-adaptive feature point descriptor is generated to eliminate the phenomenon of increased descriptor distance caused by changes in ambient illumination.

5. The method of claim 4, wherein, The process of adjusting the geometric constraint weights in the 3D reconstruction process based on the implicit correlation parameter model set includes: Obtain the classification results and geometric parameters of non-standard geometric shapes in the target scene. The non-standard geometric shapes include at least one of curved walls, irregular decorative surfaces, stepped facades, and inclined support columns. The geometric parameters include at least the radius of curvature of the curved area, the height difference of the stepped surfaces, and the tilt angle of the facade. Based on the classification results of the non-standard geometric shapes and the geometric parameters, the geometric type of the region to be reconstructed in the target scene and the corresponding initial constraint smoothness parameters are determined. Based on the solar altitude angle and wall orientation at the current image acquisition time, the association model between the light source direction and the scene shadow distribution in the implicit association parameter model set is called to calculate the shadow distribution characteristics of the area to be reconstructed under the current lighting conditions. The shadow distribution characteristics include at least shadow coverage and shadow boundary position. Based on the shadow distribution characteristics, the initial constraint smoothness parameters are adjusted to generate optimized constraint smoothness parameters that match the current lighting conditions and the non-standard geometric shape. Based on the optimized constraint smoothness parameters, surface fitting and surface optimization are performed on the point cloud data of the region to be reconstructed to generate the reconstructed surface with the non-standard geometric shape.

6. The method of claim 5, wherein, Also includes: Identify the curved surface regions in the target scene and extract the radius of curvature of the curved surface regions; Adjust the smoothness parameter of the surface fitting based on the surface curvature radius and the current lighting conditions; In response to the surface curvature radius being less than a preset curvature threshold and under side lighting conditions, the smoothness parameter is reduced to preserve surface detail features, and the weight of the line segment constraint is increased to constrain the fitting direction of the surface region. In response to the surface curvature radius being greater than or equal to the preset curvature threshold and under positive top lighting conditions, the weights of the smoothness parameter and the patch normal constraint are increased to suppress surface undulations caused by noise and ensure the continuity of the surface normal. Based on the dynamically adjusted smoothness parameters, the weights of the line segment constraints, and the weights of the patch normal constraints, point cloud optimization and surface fitting are performed on the surface region.

7. The method of claim 1, wherein, It also includes establishing spatiotemporal tags for occlusions in the target scene, including the time of occurrence and the degree of impact; During the feature matching stage, the matching weight of feature points in the occluded area is reduced based on the spatiotemporal label. In the multi-view fusion stage, complementary image information collected at different times is used to compensate and reconstruct the occluded area.

8. The method according to claim 7, wherein, The process of compensating and reconstructing the occluded area using complementary image information acquired at different time periods during the multi-view fusion stage includes: The image sequences collected from each viewpoint and time period in the target scene are obtained, along with the collection timestamp and spatiotemporal label corresponding to each image; the spatiotemporal label includes at least the type of occlusion, the time period of occurrence, and the degree of impact. Based on the spatiotemporal tags, occlusion regions are marked for each image in the image sequence to generate an occlusion mask map. The occlusion mask map is used to mark the pixel regions in the image that are covered by occlusion objects. Using the 3D model of the target scene as a spatial reference, images acquired at each time period are projected onto the corresponding surface regions of the 3D model, and the visibility status of each surface region in each image is determined based on the occlusion mask map; the visibility status includes visible, occluded, and partially visible. For a target surface region that is marked as occluded in the first image acquired at the first time and marked as visible in the second image acquired at the second time, extract the texture information and geometric constraint information of the corresponding region in the second image; The texture information and geometric constraint information are mapped to the corresponding positions of the target surface region in the 3D model according to the projection mapping relationship, and compensation texture and compensation geometric data are generated. Based on this, surface filling and smoothing optimization are performed on the target surface region to obtain the occlusion-compensated 3D model.

9. A campus non-standard wall surface three-dimensional reconstruction and parameter calibration system, characterized in that, The system, applied to the method for three-dimensional reconstruction and parameter calibration of non-standard campus walls as described in any one of claims 1-8, specifically includes: The data acquisition module is used to acquire image data of the target scene and corresponding environmental state data; The correlation analysis module is used to analyze the set of implicit correlation parameter models between the environmental state data and image features. The implicit correlation relationship includes at least one of the following: a nonlinear mapping model between ambient light intensity and global image brightness, a statistical correlation model between ambient light intensity and feature point detection repetition rate, and a correlation model between light source direction and scene shadow distribution. This model is used to characterize the influence of different environmental states on the performance of image features. The parameter dynamic adjustment module is used to adjust at least one of the feature extraction parameters and geometric constraint weights in the 3D reconstruction process based on the implicit correlation parameter model set. A 3D reconstruction module is used to perform 3D reconstruction on the image data based on at least one of the adjusted feature extraction parameters and the geometric constraint weights to obtain a 3D model of the target scene.

10. The campus non-standard wall surface three-dimensional reconstruction and parameter calibration system according to claim 9, characterized in that, It also includes an illumination adaptive calibration and correction module, which is used for: In response to the camera calibration start signal, the environmental state data is recorded synchronously; Based on the set of implicit correlation parameter models, the reliability of the calibration board corner detection is evaluated; In response to the calibration result optimization start signal, the weight of low-confidence calibration images is removed or reduced, and camera calibration parameters with confidence evaluation are output.

Images