A caisson maintenance method and system

Abstract

This application relates to a caisson maintenance method and system. The method includes collecting multimodal data of the caisson; the multimodal data includes three-dimensional point cloud data, multispectral image data, and macro image data; using the multimodal data to calculate the center point and symmetry axis of the caisson; aligning the three-dimensional point cloud data, multispectral image data, and macro image data using the center point and symmetry axis of the caisson; performing layered data fusion on the three-dimensional point cloud data, multispectral image data, and macro image data to obtain a fused caisson model to be repaired; extracting missing pattern data from the caisson model to be repaired, inputting the missing pattern data into a pre-trained domain-adaptive generative adversarial network to generate repair suggestions, and then comparing the repair suggestions against a knowledge graph in real time to constrain the repair suggestions; and repairing the caisson using the constrained repair suggestions. This application achieves high-precision repair of caisson structures.

Description

Technical Field

[0001] The present application relates to the technical field of caissons, and in particular to a caisson maintenance method and system. Background Art

[0002] The caisson is a unique interior ceiling decorative structure in ancient Chinese architecture, commonly found in important buildings such as palaces and temples. It not only has extremely high artistic value, but also carries a wealth of historical and cultural information. Usually constructed of wood and painted with exquisite colors on the surface, the caisson has a complex structure and gorgeous decorations, making it an important physical material for studying ancient Chinese architectural art. Over time, the caisson suffers from natural aging, environmental erosion, and human damage, resulting in structural damage, fading of the colors, and other problems, urgently requiring effective protection and restoration measures. Existing caisson protection and restoration methods mainly rely on traditional means, such as manual mapping and single technologies (such as laser scanning or photogrammetry). However, these methods suffer from insufficient accuracy and low efficiency. Summary of the Invention

[0003] The object of the present invention is to provide a caisson maintenance method and system to partially solve or alleviate the above-mentioned deficiencies in the prior art and to achieve high-efficiency and high-precision repair effects on the caisson structure.

[0004] In order to solve the above-mentioned technical problems, the present invention specifically adopts the following technical solutions:

[0005] A first aspect of the present invention is to provide a caisson maintenance method, characterized in that it includes the following steps:

[0006] Collecting multimodal data of the caisson; the multimodal data includes three-dimensional point cloud data, multispectral image data, and macro image data;

[0007] Calculating the center point and symmetry axis of the caisson using the multimodal data;

[0008] aligning the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson;

[0009] Performing layered data fusion on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain a fused caisson model to be restored;

[0010] Extracting missing pattern data from the caisson model to be repaired, inputting the missing pattern data into a pre-trained domain-adaptive generative adversarial network to generate repair suggestions, and comparing the knowledge graph in real time to constrain the repair suggestions;

[0011] The caisson was repaired using the constrained repair suggestions.

[0012] By adopting the above solution, through adaptive multimodal data fusion algorithm, combined with lidar, multispectral imaging and macro photography data, millimeter-level precision modeling and restoration of the caisson structure can be achieved; using the cultural context-driven AI restoration framework, it is ensured that the restoration plan adheres to historical style and architectural regulations, avoids restorative damage, and protects the authenticity and integrity of cultural heritage.

[0013] Furthermore, the calculating the center point and the symmetry axis of the caisson using the multimodal data includes:

[0014] The three-dimensional point cloud data is acquired, the center point of the caisson is obtained by calculating the centroid of the three-dimensional point cloud, and the symmetry axis of the point cloud is obtained by a principal component analysis method.

[0015] Furthermore, the aligning of the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson includes:

[0016] Performing feature extraction on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain key feature points and feature vectors of the multimodal data;

[0017] Calculating the distances between the feature vectors of the three-dimensional point cloud data, the multispectral image data, and the macro image data, and screening matching pairs using a nearest neighbor search;

[0018] Taking the center point of the caisson as the reference, the matching key feature points are aligned using the axis of symmetry.

[0019] Furthermore, performing layered data fusion on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain the fused caisson model to be restored includes:

[0020] Extracting geometric structural features of the caisson from the three-dimensional point cloud data; extracting texture distribution features, pattern turning points, and spectral material property data of the painted layer from the multispectral image data; and extracting microscopic detail features from the macro image data;

[0021] The geometric structure features are used to construct a three-dimensional model of the caisson structure layer; the texture distribution features and pattern turning points in the multispectral image data are combined with the microscopic detail features in the macro image data to construct a texture model of the painted layer; the microscopic detail features and spectral material property data are aligned to the three-dimensional model to generate a labeled material layer.

[0022] Furthermore, the steps include:

[0023] Obtain a restored caisson model, generate incremental changes by comparing the restored model with the original model point by point, and mark the incremental changes on the restored model using a visualization tool.

[0024] Furthermore, the steps include:

[0025] Export the restored caisson model to a format supported by AR devices;

[0026] Import the exported model into the software platform of the AR device;

[0027] The incremental changes are projected onto the caisson body through the AR device, and the model is repaired through interaction with the AR model through gesture operation.

[0028] Furthermore, the steps include:

[0029] Real-time monitoring data in the caisson is obtained, and the deformation risk of the caisson is predicted using the real-time monitoring data and a reinforcement plan is proposed.

[0030] Furthermore, the acquiring of real-time monitoring data in the caisson, and using the real-time monitoring data to predict deformation risk of the caisson and recommend a reinforcement plan include:

[0031] Regularly collect environmental data and structural deformation data based on real-time monitoring equipment;

[0032] Transmit the collected data to the central control system via wireless network;

[0033] The structural deformation data and real-time environmental data are input into the LSTM time series analysis model to predict the deformation trend, the deformation risk level is evaluated according to the predicted deformation trend, and a reinforcement plan is generated according to the deformation risk level.

[0034] In a second aspect, the present application further discloses a caisson maintenance system, comprising:

[0035] A multimodal data acquisition module configured to acquire multimodal data of the caisson; the multimodal data includes three-dimensional point cloud data, multispectral image data, and macro image data;

[0036] A geometric calculation module configured to calculate the center point and symmetry axis of the caisson using the multimodal data;

[0037] a data alignment module configured to align the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson;

[0038] a layered fusion module configured to perform layered data fusion on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain a fused caisson model to be restored;

[0039] A missing pattern completion module is configured to extract missing pattern data from the caisson model to be repaired, input the missing pattern data into a pre-trained domain-adaptive generative adversarial network to generate repair suggestions, and compare the knowledge graph in real time to constrain the repair suggestions;

[0040] A repair module configured to repair the caisson using the constrained repair suggestions;

[0041] Among them, the layered fusion module is specifically used to extract the geometric structural features of the caisson from the three-dimensional point cloud data; extract the texture distribution features, pattern turning points and spectral material property data of the painted layer from the multispectral image data; extract microscopic detail features from the macro image data; use the geometric structural features to construct a three-dimensional model of the caisson structure layer; combine the texture distribution features and pattern turning points in the multispectral image data with the microscopic detail features in the macro image data to construct a texture model of the painted layer; align the microscopic detail features and spectral material property data into the three-dimensional model to generate a labeled material layer.

[0042] Beneficial effects:

[0043] 1. Through adaptive multimodal data fusion algorithms, combined with lidar, multispectral imaging and macro photography data, millimeter-level precision modeling and restoration of caisson structures are achieved.

[0044] 2. Utilize a cultural context-driven AI restoration framework to ensure that restoration plans adhere to historical styles and architectural regulations, avoid restorative damage, and protect the authenticity and integrity of cultural heritage.

[0045] 3. Develop a deformation-environment coupling analysis model to predict the caisson deformation trend by real-time monitoring of the relationship between environmental parameters and structural deformation, and provide timely warnings for high-risk areas to achieve preventive protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the embodiments or the description of the prior art. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the various elements or parts are not necessarily drawn according to the actual scale. Obviously, the drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without inventive work.

[0047] Figure 1 This is a flow chart of a caisson maintenance method of the present application.

[0048] Figure 2 It is a schematic diagram of collecting multimodal data in one embodiment of the present application.

[0049] Figure 3 This is a schematic diagram of the missing pattern repair process in one embodiment of the present application.

[0050] Figure 4 It is a schematic diagram of the module structure of the caisson maintenance system in one embodiment of the present application. DETAILED DESCRIPTION

[0051] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] Herein, suffixes such as "module," "component," or "unit" used to represent elements are only used to facilitate description of the present invention and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0053] As used herein, terms such as "upper," "lower," "inner," "outer," "front," "back," "one end," and "the other end" indicate positions or locations based on those shown in the accompanying drawings. These terms are intended solely to facilitate and simplify the description of the present invention and are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0054] As used herein, unless otherwise expressly specified or limited, the terms "installed," "provided with," and "connected" should be understood broadly. For example, "connected" may refer to a fixed connection, a detachable connection, or an integral connection; it may refer to a mechanical connection, a direct connection, an indirect connection via an intermediate medium, or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention on a case-by-case basis.

[0055] As used herein, "and / or" includes any and all combinations of one or more of the associated listed items.

[0056] Herein, "plurality" means two or more than two, ie, it includes two, three, four, five, etc.

[0057] Figure 1A flowchart of a caisson maintenance method of the present application is shown, referring to Figure 1 , the method specifically comprises the following steps:

[0058] Step 1: Collect multimodal data of the caisson; the multimodal data includes three-dimensional point cloud data, multispectral image data, and macro image data;

[0059] refer to Figure 2 In this specific embodiment, point cloud data is primarily acquired using LiDAR equipment and then transmitted to the control center via a wireless transmission module. This creates a 3D outline of the caisson's overall structure and key points, serving as a basis for subsequent deformation assessment. LiDAR scanning: A handheld LiDAR device scans the caisson with millimeter-level accuracy, generating 3D point cloud data (recorded as version L1).

[0060] In a specific embodiment, multispectral image acquisition primarily involves using multispectral cameras to capture real-time external image information of the caisson. This information is compared with data collected by LiDAR to determine overall deformation and damage at key points. This image acquisition equipment is typically placed directly on the ground beneath the caisson and connected to the control center via a wired connection. The multispectral camera also captures the pigment composition and disease distribution of the caisson's painted layer (recorded as version M1). Semantic annotation: areas of discoloration (such as oxidation of cinnabar pigment) and areas of flaking are annotated.

[0061] The SIFT algorithm can be used to determine whether the caisson has deformed, and the Canny edge detection algorithm can be used to detect the edges of the caisson. In addition, a mechanism for automatically triggering detection based on meteorological factors has been added, which will automatically trigger defect detection in extreme weather conditions.

[0062] In a specific embodiment, macro image acquisition is mainly achieved through macro camera equipment. In addition to collecting information on key nodes of the caisson, it is also necessary to focus on observing subtle changes in the patterns, cracks, and color loss of the caisson. These devices need to be placed at the key observation targets of the caisson. In order to avoid damage to the building itself due to the installation of network cables, wireless transmission is used. Macro photography: Take high-resolution macro photos (recorded as version C1) to capture pattern details (such as dragon scales and cracks <0.1mm). In addition, in the process of detecting information such as patterns and cracks, deep learning methods can also be used to train the input pattern and other information through the GAN network to determine abnormal situations.

[0063] The image information of macro photography includes two parts: one is the image of the appearance of the caisson, which is detected by the camera placed under the caisson and is only responsible for the damage and deformation of the external contour; the other is the image detection information of the microscopic pattern cracks placed inside the caisson, which is used to detect details.

[0064] Step 2: Calculate the center point and symmetry axis of the caisson using multimodal data;

[0065] In a specific embodiment, calculating the center point and symmetry axis of the caisson using multimodal data includes:

[0066] The three-dimensional point cloud data was obtained, the center point of the caisson was obtained by calculating the centroid of the three-dimensional point cloud, and the symmetry axis of the point cloud was obtained by principal component analysis.

[0067] In a specific embodiment, the formula for obtaining the center point of the caisson by calculating the centroid of the three-dimensional point cloud is as follows:

[0068]

[0069] Where C represents the coordinates of the calculated geometric center point; N represents the total number of points in the point cloud data; x i 、y i 、z i They represent the x, y, and z coordinates of the i-th point in three-dimensional space.

[0070] In addition, the center point position can be further adjusted by using the Iterative Closest Point (ICP) algorithm or other optimization methods to make it more consistent with the actual geometric center of the caisson:

[0071]

[0072] Among them, C opt Indicates the center point coordinates obtained after optimization. ICP is the iterative closest point algorithm used to optimize the center point position. L1" represents the point cloud data after preprocessing (such as denoising, downsampling, etc.).

[0073] In a specific embodiment, the specific steps of obtaining the symmetry axis of the point cloud by the principal component analysis method are as follows:

[0074] Principal Component Analysis (PCA): By performing principal component analysis on point cloud data, the main direction of the point cloud data, that is, the direction of the symmetry axis, is found.

[0075] First, calculate the covariance matrix:

[0076]

[0077] Among them, P L Represents the coordinate vector of the i-th point in the point cloud.

[0078] Compute eigenvalues and eigenvectors:

[0079]

[0080] represents the j-th eigenvector of the matrix Σ, which is a column vector; λj represents the j-th eigenvalue of the matrix Σ;

[0081] The eigenvector corresponding to the largest eigenvalue is chosen as the direction of the symmetry axis:

[0082]

[0083] in, represents the eigenvector with the largest eigenvalue.

[0084] Combined with the optimized center point C opt and direction vector , represents the axis of symmetry A.

[0085]

[0086] Here, t is a real number parameter.

[0087] In a specific embodiment, point cloud data is used to obtain the hierarchical boundaries of a caisson (e.g., the bracket layer and the well core layer). Geometric analysis and image processing techniques can be used to extract features from the point cloud data. For example, edge detection algorithms (e.g., the Canny edge detector) are applied to identify the edges of the structure, which may correspond to the boundaries between different layers. Planes are fitted using methods such as the RANSAC algorithm to identify possible hierarchical structures. For example, the bracket layer and the well core layer may appear as different planes.

[0088] In one specific embodiment, because the caisson is located at the top of a building component, it is not convenient to manually add positioning points. Therefore, a laser emission source is used to locate information about the key locations of the caisson, extracting basic point cloud data such as the center point, symmetry line, and edge information of the caisson. After this information is collected, it is transmitted to the control center (software) via a wireless transmission module. LiDAR is used to obtain key locations of the caisson. First, the target points to be captured are manually determined, such as the center point, symmetry line, and edge position. Then, the point cloud data of these key points is obtained using LiDAR, and then transmitted to the control center via the wireless transmission module.

[0089] The key targets here mainly include:

[0090] 1) Center point: used to confirm the center of the caisson. If the building is displaced and the position of the caisson changes, the center point can be used for identification.

[0091] 2) Symmetry lines: Collect the horizontal and vertical symmetry lines of the caisson, so that problems can be discovered when asymmetric deformation occurs in the caisson itself;

[0092] 3) Edge position: These data mainly refer to the position of the caisson frame. If the caisson frame is loose, the edge position can be used to identify the abnormality.

[0093] 4) Mortise and tenon joints: mainly for the positioning of some key mortise and tenon components. If there is damage or loosening of the mortise and tenon, it can be judged by the changes in the mortise and tenon joints;

[0094] 5) Pattern turning points: mainly used to identify some key pattern positions. These data can be used to determine the relative positions of key components. If pattern deformation occurs, abnormalities can be detected through these relative positions.

[0095] Step 3: Use the center point and symmetry axis of the caisson to align the three-dimensional point cloud data, multispectral image data, and macro image data.

[0096] In a specific embodiment, aligning the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson includes:

[0097] Perform feature extraction on 3D point cloud data, multispectral image data, and macro image data to obtain key feature points and feature vectors of multimodal data;

[0098] Calculate the distance between the feature vectors of 3D point cloud data, multispectral image data, and macro image data, and use nearest neighbor search to select matching pairs.

[0099] Taking the center point of the caisson as the reference, the matching key feature points are aligned using the axis of symmetry.

[0100] In a specific embodiment, during centrosymmetric constrained alignment, the algorithm may search for key feature points (such as mortise and tenon joints and pattern turning points) and attempt to find corresponding points between different data sources. By matching these feature points, the algorithm can determine the relative position and orientation between the different data sources, thereby achieving more precise alignment. This method is particularly suitable for processing ancient architectural elements with repetitive patterns or structural symmetry, such as the caisson ceiling in this application.

[0101] Step 4: Perform layered data fusion on the three-dimensional point cloud data, multispectral image data, and macro image data to obtain a fused caisson model to be restored.

[0102] In a specific embodiment, performing layered data fusion on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain the fused caisson model to be restored includes:

[0103] Extract the geometric structural features of the caisson from 3D point cloud data; extract the texture distribution characteristics, pattern turning points, and spectral material property data of the painted layer from multispectral image data; and extract microscopic detail features from macro image data.

[0104] The three-dimensional model of the caisson structure layer is constructed using geometric structural features. The texture distribution features and pattern turning points in the multispectral image data are combined with the microscopic detail features in the macro image data to construct a texture model of the painted layer. The microscopic detail features and spectral material property data are aligned into the three-dimensional model to generate a labeled material layer.

[0105] Step 5: Extract the missing pattern data from the caisson model to be repaired, input the missing pattern data into the pre-trained domain adaptive generative adversarial network to generate repair suggestions, and compare the knowledge graph in real time to constrain the repair suggestions. Figure 3 .

[0106] In a specific embodiment, the traditional method tends to ignore the hierarchical regulations of ancient architectural paintings (such as "Hexi colored paintings" are only used for royal buildings), resulting in restoration results that violate historical authenticity. Figure 3 In step S05, a pre-trained domain-adaptive generative adversarial network (DA-GAN) is used to generate repair suggestions and compare them with knowledge graph constraints in real time. The knowledge base of Ming and Qing Dynasty official architectural paintings is accessed to verify the pattern style of the repair plan (such as the scale density of the "Golden Dragon and Seal" and the structural logic of the brackets). Figure 3 (Step S06 in the above text).

[0107] Pattern knowledge graphs: For example, the rules of Ming and Qing dynasty official architectural murals (such as pattern type, color matching, and composition ratio) are encoded into a structured knowledge base. For example, in the "Golden Dragon and Seal" mural, the dragon scale density is 8-12 pieces / cm, and the head orientation must conform to the principle of "centripetality." The completion of bracket patterns must follow the structural logic of "single-curved, single-raised, and five-stepped."

[0108] In another specific embodiment, a pre-built knowledge graph is used to train a domain-adversarial generative network (DA-GAN). When training a DA-GAN, the loss function generally includes two parts: one is the traditional generative adversarial network (GAN) loss, and the other is the domain-adversarial loss. The latter uses the knowledge graph to enhance the discriminator's ability to distinguish between the source domain and the target domain:

[0109] 1. Traditional GAN loss:

[0110] The discriminator D attempts to distinguish between real samples and generated samples, and its loss function can be expressed as:

[0111]

[0112] in, Represents the real data distribution The expectation of sample x under ; logD(x) represents the logarithmic probability that the discriminator D judges that the sample x is a true sample; Represents the prior distribution The expectation of the noise z under the condition of G(z) represents the sample generated by the generator G according to the noise z. The prior distribution of the noise representing the generator input, usually a uniform distribution or a Gaussian distribution.

[0113] 2. Domain confrontation loss:

[0114] The domain adversarial loss aims to make it impossible for the discriminator to distinguish whether the sample comes from the source domain or the target domain. This can be achieved by maximizing the domain classification loss of the source and target domain samples on the discriminator output:

[0115]

[0116] in, Represents the distribution of source domain samples The expectation of the sample x under logD domain (x) represents the domain classification part D of the discriminator D domain The logarithmic probability that the source domain sample x belongs to the source domain. Table represents the distribution of target domain samples The following sample expectations. Denotes the domain classification part D of the discriminator D domain For target domain samples The log probability of not belonging to the source domain (i.e., belonging to the target domain).

[0117] 3. Knowledge graph enhancement loss:

[0118] Leverage the structural information in the knowledge graph to enhance the domain adversarial loss. This can be achieved by maximizing the distance between the embeddings of the source and target domain entities in the knowledge graph:

[0119]

[0120] in, Indicates that entity e belongs to the source domain entity set and the target domain entity set The union of . D kg (e) represents the embedding representation of entity e by the knowledge graph embedding part of the discriminator D. T(e) represents the corresponding entity of entity e in the target domain. Here we assume that there is a mapping T that can map entities in the source domain to corresponding entities in the target domain.

[0121] 4. Total loss function:

[0122] Combining the above three losses, we get the total loss function of the discriminator:

[0123]

[0124] Among them, λ adv and λ kg is a weight parameter used to balance the contribution of different loss terms.

[0125] Based on an AI restoration method driven by cultural context, a knowledge graph of caisson patterns was constructed (incorporating the regulations on colored paintings of official buildings in the Ming and Qing dynasties), and a domain-adaptive generative adversarial network (DA-GAN) was trained. When completing incomplete patterns, it automatically follows the historical style (such as the density and direction of the scales of the "Golden Dragon and Seal" pattern) to avoid "style conflicts."

[0126] Step 6: Use the constrained repair suggestions to repair the caisson.

[0127] In a specific embodiment, step six further includes:

[0128] Obtain the repaired caisson model, generate incremental changes by comparing the differences between the repaired model and the original model point by point, and use visualization tools to mark the incremental changes on the repaired model.

[0129] For example, a new version is generated after each fix (e.g. Figure 3 In step S07 of step B2), the system automatically marks incremental changes (e.g., repaired areas are highlighted in green) and supports users to compare historical versions (B1 vs. B2).

[0130] In a specific embodiment, step six further includes:

[0131] Export the restored caisson model to a format supported by AR devices;

[0132] Import the exported model into the software platform of the AR device;

[0133] The incremental changes are projected onto the caisson body through the AR device, and the model is repaired through gesture operation and interaction with the AR model.

[0134] Through this AR interactive display method, the fusion model is imported into the AR device (such as Hololens), and the user can use gestures to "disassemble" the caisson structure layer to view the repair effect.

[0135] In a specific embodiment, for example:

[0136] 1. Data collection: Laser scanning revealed a 0.2 mm crack in the mortise and tenon joint (L1), and multispectral imaging detected fading of the painted brackets (M1).

[0137] 2. Adaptive alignment: The system automatically aligns cracks and faded areas based on the axis of symmetry to determine the repair priority.

[0138] 3. Intelligent restoration: DA-GAN generates a solution to complete the bracket arch pattern, and the knowledge graph verifies that it complies with the "single-curved, single-raised, five-step" regulation.

[0139] 4. AR verification: The restoration plan is projected onto the caisson through AR, and the restoration engineer will execute it after confirming that it is correct.

[0140] In a specific embodiment, the method further includes step seven: obtaining real-time monitoring data in the caisson, using the real-time monitoring data to predict the deformation risk of the caisson and recommending a reinforcement plan.

[0141] In a specific embodiment, obtaining real-time monitoring data in the caisson, using the real-time monitoring data to predict deformation risks of the caisson and recommending a reinforcement plan includes:

[0142] Regularly collect environmental data and structural deformation data based on real-time monitoring equipment;

[0143] Transmit the collected data to the central control system via wireless network;

[0144] Structural deformation data and real-time environmental data are input into the LSTM time series analysis model to predict deformation trends. The deformation risk level is evaluated based on the predicted deformation trend, and a reinforcement plan is generated based on the deformation risk level.

[0145] In a specific embodiment, this dynamic risk warning mechanism is used to develop a deformation-environment coupling analysis model, predict the caisson deformation trend through time series analysis (such as LSTM network), and locate high-risk areas (such as the fatigue threshold of the mortise and tenon joint in the northwest corner under humidity cycle).

[0146] In a specific embodiment, for example, if deformation of the exterior of the caisson is detected through edge detection and deformation monitoring algorithms during multispectral image acquisition, different warning messages will be given according to the degree of deformation; if possible risks such as patterns and cracks are identified through deep learning during macro image acquisition, the risk type and level of the specific pattern will be given.

[0147] In a specific embodiment, if the meteorological environment deteriorates, such as being in a humid environment for a long time, continuous high temperature, strong wind and other extreme weather conditions, damage detection and warning will be automatically triggered.

[0148] In a specific embodiment, the factors affecting the deformation of the caisson mainly include:

[0149] 1) External climatic factors, including internal and external temperature and humidity, light, wind and sand, etc., will affect the wooden structure and painting of the caisson body.

[0150] 2) Structural changes of the caisson itself due to structural strain, physical factors, etc.

[0151] Therefore, the monitoring data collection of the caisson includes the external environment data collection and the internal caisson body data collection.

[0152] External climate information collection:

[0153] The collection of external climate factors mainly relies on temperature and humidity sensors, light sensors, wind sensors and other sensors to collect meteorological information. These sensors are distributed inside and outside the caisson building and are deployed at multiple points to collect meteorological factors at different heights and locations.

[0154] Caisson body information collection:

[0155] The collection of information about the caisson itself is mainly done with the help of cameras. Multiple cameras will be set up inside the building to collect image information of key parts of the caisson.

[0156] After being collected, the above-mentioned external climate information and caisson body information will be transmitted to a unified controller through network equipment for secondary processing and treatment.

[0157] In a specific embodiment, embedded flexible sensors are set in the caisson. For example, flexible strain sensors are deployed in hidden places in the caisson (such as mortise and tenon joints), which can bend with the deformation of the wood to achieve non-invasive monitoring (traditional rigid sensors require drilling and installation, which damages the cultural relic itself).

[0158] In a specific embodiment, step seven also includes: using real-time monitoring equipment to collect meteorological data and caisson status data of the caisson environment, using the meteorological data and caisson status data to calculate the meteorological disturbance factor, and using the meteorological disturbance factor to correct the deformation risk level.

[0159] In a specific embodiment, after collecting the corresponding meteorological information, it is necessary to construct a disturbance factor based on some current meteorological information. That is, when judging the micro-deformations such as the external contour or pattern of the caisson, it is necessary to consider the influence of current climate factors. If the current temperature is too high, it is necessary to consider the influence of thermal expansion and contraction of the wooden structure and adjust the corresponding disturbance coefficient in the fault identification process. If the wind is strong, the disturbance coefficient in the model is adjusted according to the specific wind level.

[0160] This application, through a differentiated innovation approach, innovatively proposes multimodal data fusion modeling—proposing a heterogeneous data alignment algorithm combining lidar, multispectral imaging, and macro photography; cultural context-driven AI restoration—building a knowledge graph of caisson patterns (incorporating Ming and Qing dynasty official architectural painted regulations) and training a domain-adaptive generative adversarial network (DA-GAN); and a dynamic risk warning mechanism—developing a deformation-environment coupling analysis model to predict caisson deformation trends through time series analysis (such as LSTM networks) and locate high-risk areas (such as the fatigue threshold of the mortise and tenon joints in the northwest corner under humidity cycles). Through three major innovations: structure-driven alignment, culturally embedded restoration, and dynamic closed-loop updating, this application breaks through the bottlenecks of data fragmentation and style distortion in traditional digital preservation, providing a comprehensive solution for wooden heritage structures such as caissons, from "rescue restoration" to "preventive management," with both academic value and industrial potential.

[0161] Further references Figure 4 As an implementation of the above-mentioned method, the present application provides an embodiment of a caisson maintenance system. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0162] refer to Figure 4 , a caisson maintenance system, comprising:

[0163] The multimodal data acquisition module 101 is configured to acquire multimodal data of the caisson; the multimodal data includes three-dimensional point cloud data, multispectral image data, and macro image data;

[0164] A geometric calculation module 102 is configured to calculate the center point and symmetry axis of the caisson using the multimodal data;

[0165] A data alignment module 103 is configured to align the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson;

[0166] The layered fusion module 104 is configured to perform layered data fusion on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain a fused caisson model to be restored. Specifically, the layered fusion module is configured to extract geometric structural features of the caisson from the three-dimensional point cloud data; extract texture distribution features, pattern turning points, and spectral material property data of the painted layer from the multispectral image data; extract microscopic detail features from the macro image data; construct a three-dimensional model of the caisson structure layer using the geometric structural features; combine the texture distribution features and pattern turning points in the multispectral image data with the microscopic detail features in the macro image data to construct a texture model of the painted layer; and align the microscopic detail features and the spectral material property data to the three-dimensional model to generate a labeled material layer.

[0167] The missing pattern completion module 105 is configured to extract missing pattern data from the caisson model to be repaired, input the missing pattern data into a pre-trained domain-adaptive generative adversarial network to generate repair suggestions, and compare the knowledge graph in real time to constrain the repair suggestions;

[0168] The repair module 106 is configured to repair the caisson using the constrained repair suggestions.

[0169] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component and / or installed from removable media. When executed by a central processing unit (CPU), the computer program performs the aforementioned functions defined in the methods of the present application.

[0170] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device implements the following when executing. Figure 1 The method shown in .

[0171] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0172] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a computer terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0173] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A caisson maintenance method, characterized in that: Including steps: Collecting multimodal data of the caisson; the multimodal data includes three-dimensional point cloud data, multispectral image data, and macro image data; Calculating the center point and symmetry axis of the caisson using the multimodal data; aligning the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson; Performing layered data fusion on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain a fused caisson model to be restored; Extracting missing pattern data from the caisson model to be repaired, inputting the missing pattern data into a pre-trained domain-adaptive generative adversarial network to generate repair suggestions, and comparing the knowledge graph in real time to constrain the repair suggestions; Repair the caisson using the constrained repair suggestions; The three-dimensional point cloud data, multispectral image data and macro image data are subjected to layered data fusion to obtain the fused caisson model to be restored, which includes: Extracting geometric structural features of the caisson from the three-dimensional point cloud data; extracting texture distribution features, pattern turning points, and spectral material property data of the painted layer from the multispectral image data; and extracting microscopic detail features from the macro image data; The geometric structure features are used to construct a three-dimensional model of the caisson structure layer; the texture distribution features and pattern turning points in the multispectral image data are combined with the microscopic detail features in the macro image data to construct a texture model of the painted layer; the microscopic detail features and spectral material property data are aligned to the three-dimensional model to generate a labeled material layer.

2. A caisson maintenance method according to claim 1, characterized in that: The calculating the center point and the symmetry axis of the caisson by using the multimodal data includes: The three-dimensional point cloud data is acquired, the center point of the caisson is obtained by calculating the centroid of the three-dimensional point cloud, and the symmetry axis of the point cloud is obtained by a principal component analysis method.

3. A caisson maintenance method according to claim 2, characterized in that The method of aligning the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson includes: Performing feature extraction on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain key feature points and feature vectors of the multimodal data; Calculating the distances between the feature vectors of the three-dimensional point cloud data, the multispectral image data, and the macro image data, and screening matching pairs using a nearest neighbor search; Taking the center point of the caisson as the reference, the matching key feature points are aligned using the axis of symmetry.

4. A caisson maintenance method according to claim 3, characterized in that: The calculation formula for calculating the centroid of the three-dimensional point cloud to obtain the center point of the caisson is: ， Where C represents the coordinates of the calculated geometric center point; N represents the total number of points in the point cloud data; x i 、y i 、z i They represent the x, y, and z coordinate values of the i-th point in three-dimensional space.

5. A caisson maintenance method according to any one of claims 1 to 4, characterized in that: Also includes the steps: Obtain a restored caisson model, generate incremental changes by comparing the restored model with the original model point by point, and mark the incremental changes on the restored model using a visualization tool.

6. A caisson maintenance method according to claim 5, characterized in that: Also includes the steps: Export the restored caisson model to a format supported by AR devices; Import the exported model into the software platform of the AR device; The incremental changes are projected onto the caisson body through the AR device, and the model is repaired through interaction with the AR model through gesture operation.

7. A caisson maintenance method according to any one of claims 1 to 4, characterized in that: Also includes the steps: Real-time monitoring data in the caisson is obtained, and the deformation risk of the caisson is predicted using the real-time monitoring data and a reinforcement plan is proposed.

8. A caisson maintenance method according to claim 7, characterized in that: The acquiring of real-time monitoring data in the caisson, and using the real-time monitoring data to predict deformation risk of the caisson and recommend a reinforcement plan include: Regularly collect environmental data and structural deformation data based on real-time monitoring equipment; Transmit the collected data to the central control system via wireless network; The structural deformation data and real-time environmental data are input into the LSTM time series analysis model to predict the deformation trend, the deformation risk level is evaluated according to the predicted deformation trend, and a reinforcement plan is generated according to the deformation risk level.

9. A caisson maintenance method according to claim 8, characterized in that: Also includes the steps: Real-time monitoring equipment is used to collect meteorological data and caisson status data of the environment where the caisson is located, the meteorological data and caisson status data are used to calculate the meteorological disturbance factor, and the meteorological disturbance factor is used to correct the deformation risk level.

10. A caisson maintenance system, characterized in that: include: A multimodal data acquisition module configured to acquire multimodal data of the caisson; the multimodal data includes three-dimensional point cloud data, multispectral image data, and macro image data; A geometric calculation module configured to calculate the center point and symmetry axis of the caisson using the multimodal data; a data alignment module configured to align the three-dimensional point cloud data, the multispectral image data, and the macro image data using the center point and the symmetry axis of the caisson; a layered fusion module configured to perform layered data fusion on the three-dimensional point cloud data, the multispectral image data, and the macro image data to obtain a fused caisson model to be restored; A missing pattern completion module is configured to extract missing pattern data from the caisson model to be repaired, input the missing pattern data into a pre-trained domain-adaptive generative adversarial network to generate repair suggestions, and compare the knowledge graph in real time to constrain the repair suggestions; A repair module configured to repair the caisson using the constrained repair suggestions; The layered fusion module is specifically used to extract the geometric structural features of the caisson from the three-dimensional point cloud data; extract the texture distribution features, pattern turning points and spectral material property data of the painted layer from the multispectral image data; and extract microscopic detail features from the macro image data. The geometric structure features are used to construct a three-dimensional model of the caisson structure layer; the texture distribution features and pattern turning points in the multispectral image data are combined with the microscopic detail features in the macro image data to construct a texture model of the painted layer; the microscopic detail features and spectral material property data are aligned to the three-dimensional model to generate a labeled material layer.

Images