Caisson maintenance method and caisson maintenance system

Abstract

The invention relates to a caisson maintenance method and system. The method comprises the steps that caisson multi-modal data are collected; the multi-modal data comprises three-dimensional point cloud data, multispectral image data and microspur image data; calculating the center point and the symmetry axis of the caisson by using the multi-modal data; aligning the three-dimensional point cloud data, the multispectral image data and the microspur image data by using the center point and the symmetry axis of the caisson; performing hierarchical data fusion on the three-dimensional point cloud data, the multispectral image data and the microspur image data to obtain a fused algae well model to be repaired; extracting missing pattern data from the to-be-restored caisson model, inputting the missing pattern data into a pre-trained domain adaptive generative adversarial network to generate a restoration suggestion, and comparing the knowledge graph in real time to constrain the restoration suggestion; and repairing the caisson by using the constrained repair suggestion. According to the method, the effect of high-precision repairing of the caisson structure is achieved.

Description

Technical Field

[0001] This application relates to the technical field of caissons, and in particular to a caisson maintenance method and system. Background Art

[0002] A caisson is a unique indoor ceiling decoration structure in Chinese ancient architecture, commonly found in important buildings such as palaces and temples. It not only has extremely high artistic value but also carries rich historical and cultural information. Caissons are usually constructed of wood with delicate painted patterns on the surface. With complex structures and magnificent decorations, they are important physical materials for studying Chinese ancient architectural art. Over time, caissons have been affected by natural aging, environmental erosion, and human damage, resulting in problems such as structural damage and painted pattern fading, and effective protection and restoration measures are urgently needed. Existing caisson protection and restoration methods mainly rely on traditional means, such as manual surveying and mapping and single technologies (such as laser scanning or photogrammetry), but these methods have problems such as insufficient accuracy and low efficiency. Summary of the Invention

[0003] The purpose of the present invention is to provide a caisson maintenance method and system, which can partially solve or alleviate the above deficiencies in the prior art and can achieve high-efficiency and high-precision repair effects on caisson structures.

[0004] To solve the above-mentioned technical problems, the present invention specifically adopts the following technical solutions: In the first aspect of the present invention, there is provided a caisson maintenance method, characterized by including the steps of: Collecting multi-modal data of the caisson; the multi-modal data includes three-dimensional point cloud data, multi-spectral image data, and macro image data; Calculating the center point and symmetry axis of the caisson using the multi-modal data; Aligning the three-dimensional point cloud data, multi-spectral image data, and macro image data using the center point and symmetry axis of the caisson; Performing hierarchical data fusion on the three-dimensional point cloud data, multi-spectral image data, and macro image data to obtain a fused caisson model to be repaired; Extracting missing pattern data from the caisson model to be repaired, inputting the missing pattern data into a pre-trained domain adaptation generative adversarial network to generate repair suggestions, and comparing with a knowledge graph in real time to constrain the repair suggestions; Repairing the caisson using the constrained repair suggestions.

[0005] By adopting the above - mentioned solution, through the adaptive multi - modal data fusion algorithm, combining lidar, hyperspectral imaging, and macro - photography data, millimeter - level precision modeling and restoration of the caisson structure are achieved; by using the AI restoration framework driven by cultural context, it is ensured that the restoration plan follows historical styles and architectural regulations, avoiding restoration damage and protecting the authenticity and integrity of cultural heritage.

[0006] Further, the calculating the center point and the symmetry axis of the caisson using the multi - modal data includes: Obtain the three - dimensional point cloud data, calculate the centroid of the three - dimensional point cloud to get the center point of the caisson, and obtain the symmetry axis of the point cloud through the principal component analysis method.

[0007] Further, the aligning the three - dimensional point cloud data, hyperspectral image data, and macro - image data using the center point and the symmetry axis of the caisson includes: Extract features from the three - dimensional point cloud data, hyperspectral image data, and macro - image data respectively, and obtain the key feature points and feature vectors of the multi - modal data; Calculate the distances between the feature vectors of the three - dimensional point cloud data, hyperspectral image data, and macro - image data, and use the nearest neighbor search to screen for matching pairs; Taking the center point of the caisson as the reference, use the symmetry axis to align the matching key feature points.

[0008] Further, the performing hierarchical data fusion on the three - dimensional point cloud data, hyperspectral image data, and macro - image data to obtain the fused caisson model to be restored includes: Extract the geometric structure features of the caisson from the three - dimensional point cloud data; extract the texture distribution features, pattern turning points, and spectral material property data of the painted layer from the hyperspectral image data; extract the microscopic detail features from the macro - image data; Use the geometric structure features to construct a three - dimensional model of the caisson structure layer; combine the texture distribution features and pattern turning points in the hyperspectral image data with the microscopic detail features in the macro - image data to construct a texture model of the painted layer; align the microscopic detail features and spectral material property data into the three - dimensional model to generate a labeled material layer.

[0009] Further, it also includes the steps: Obtain the restored caisson model, generate incremental changes by comparing the differences between the restored model and the original model point by point, and use a visualization tool to label the incremental changes on the restored model.

[0010] Further, it also includes the steps: Export the restored caisson model in a format supported by AR devices; Import the exported model into the software platform of the AR device; Project the incremental changes onto the caisson body through the AR device, and perform model repair through the interaction of gesture operations with the AR model.

[0011] Furthermore, it further includes the steps: Obtain the real-time monitoring data in the caisson, use the real-time monitoring data to predict the deformation risk of the caisson, and push the reinforcement plan.

[0012] Furthermore, the obtaining the real-time monitoring data in the caisson, using the real-time monitoring data to predict the deformation risk of the caisson, and pushing the reinforcement plan includes: Regularly collect environmental data and structural deformation data based on real-time monitoring devices; Transmit the collected data to the central control system through a wireless network; Input the structural deformation data and real-time environmental data into the LSTM time series analysis model to predict the deformation trend, evaluate the deformation risk level according to the predicted deformation trend, and generate a reinforcement plan according to the deformation risk level.

[0013] In a second aspect, the present application also discloses a caisson maintenance system, including: A multi-modal data acquisition module configured to acquire caisson multi-modal data; the multi-modal data includes three-dimensional point cloud data, multi-spectral image data, and macro image data; A geometric calculation module configured to calculate the center point and symmetry axis of the caisson using the multi-modal data; A data alignment module configured to align the three-dimensional point cloud data, multi-spectral image data, and macro image data using the center point and symmetry axis of the caisson; A hierarchical fusion module configured to perform hierarchical data fusion on the three-dimensional point cloud data, multi-spectral image data, and macro image data to obtain a fused caisson model to be repaired; A missing pattern completion module configured to extract missing pattern data from the caisson model to be repaired, input the missing pattern data into a pre-trained domain adaptation generative adversarial network to generate repair suggestions, and compare with the knowledge graph in real time to constrain the repair suggestions; A repair module configured to repair the caisson using the constrained repair suggestions; Among them, the hierarchical fusion module is specifically configured to extract the geometric structure features of the caisson from the three-dimensional point cloud data; extract the texture distribution features, pattern turning points, and spectral material attribute data of the painted layer from the multispectral image data; extract the microscopic detail features from the macro image data; construct a three-dimensional model of the caisson structure layer using the geometric structure features; combine the texture distribution features and pattern turning points in the multispectral image data with the microscopic detail features in the macro image data to construct a texture model of the painted layer; and align the microscopic detail features and spectral material attribute data into the three-dimensional model to generate a labeled material layer.

[0014] Beneficial effects: 1. Through the adaptive multimodal data fusion algorithm, combining lidar, multispectral imaging, and macro photography data, millimeter-level precision modeling and repair of the caisson structure are achieved.

[0015] 2. Utilize the AI repair framework driven by cultural context to ensure that the repair plan follows historical styles and architectural regulations, avoid restorative damage, and protect the authenticity and integrity of cultural heritage.

[0016] 3. Develop a deformation-environment coupling analysis model. By monitoring the relationship between environmental parameters and structural deformation in real time, predict the deformation trend of the caisson, timely warn high-risk areas, and achieve preventive protection. Brief description of the drawings

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts do not necessarily draw according to the actual scale. Obviously, the following-described drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a flowchart of a caisson maintenance method of the present application.

[0019] Figure 2 It is a schematic diagram of collecting multimodal data in an embodiment of the present application.

[0020] Figure 3 It is a schematic diagram of the missing pattern repair process in an embodiment of the present application.

[0021] Figure 4 It is a schematic diagram of the module structure of a caisson maintenance system of the present application. Detailed implementation manners

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0023] In this article, suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of describing the present invention, and they have no specific meaning in themselves. Therefore, "module", "component", or "unit" can be used interchangeably.

[0024] In this article, terms such as "upper", "lower", "inner", "outer", "front", "rear", "one end", "the other end", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0025] In this article, unless otherwise clearly defined and limited, terms such as "installation", "provided with", "connection", etc. shall be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] In this article, "and / or" includes any and all combinations of one or more of the listed related items.

[0027] In this article, "a plurality of" means two or more, that is, it includes two, three, four, five, etc.

[0028] Figure 1 The flowchart of a method for maintaining a coffered ceiling according to the present application is shown. Refer to Figure 1 , and the method specifically includes the following steps: Step 1: Collect multi-modal data of the coffered ceiling; the multi-modal data includes three-dimensional point cloud data, multi-spectral image data, and macro image data; Refer to Figure 2, in a specific embodiment, the point cloud data is mainly obtained through a lidar device and then transmitted to the control center through a wireless transmission module. The three-dimensional contour information of the overall and key points of the caisson ceiling is drawn as the basis for subsequent deformation judgment. Lidar scanning: Use a handheld LiDAR device to scan the caisson ceiling with millimeter-level accuracy to generate three-dimensional point cloud data (recorded as version L1).

[0029] In a specific embodiment, the multi-spectral image acquisition mainly involves real-time acquisition of the external image information of the caisson ceiling through a multi-spectral camera device. This information is compared with the data collected by the lidar, mainly used to judge the overall deformation of the caisson ceiling and the damage of key nodes. Such an image acquisition device is generally directly placed on the ground under the caisson ceiling and directly connected to the control center through a wired device. And the pigment composition and disease distribution of the painted layer of the caisson ceiling are obtained through a multi-spectral camera (recorded as version M1). Semantic annotation: Annotate the faded areas (such as the oxidation of cinnabar pigment) and the peeling range.

[0030] For whether the caisson ceiling has deformed, the SIFT algorithm can be used to achieve it. For the edge detection of the caisson ceiling, the Canny edge detection algorithm is used to achieve it. In addition, a mechanism for automatically triggering detection based on meteorological factors is added. When extreme weather occurs, the defect detection will be automatically triggered.

[0031] In a specific embodiment, the macro-image acquisition is mainly achieved through a macro-camera device. In addition to collecting the information of the key nodes of the caisson ceiling, it is also necessary to focus on observing the tiny change information such as the patterns, cracks, and color shedding of the caisson ceiling. These devices need to be placed at the key observation targets of the caisson ceiling. To avoid the damage to the building itself caused by laying network cables, a wireless transmission method is adopted. Macro photography: Take high-resolution macro photos (recorded as version C1) to capture the details of the patterns (such as the scales of dragon patterns, cracks <0.1mm). In addition, in the process of detecting information such as patterns and cracks, deep learning methods can also be used to achieve it. The input information such as patterns is trained through a GAN network to judge abnormal situations.

[0032] The image information of the macro photography includes two parts: one is the image of the appearance of the caisson ceiling, that is, the image detected by the camera placed under the caisson ceiling, which is only responsible for the damage and deformation of the external contour; the other is the image detection information of the microscopic patterns and cracks inside the caisson ceiling, which is used to detect details.

[0033] Step 2: Calculate the center point and the symmetry axis of the caisson ceiling using multi-modal data; In a specific embodiment, calculating the center point and the symmetry axis of the caisson ceiling using multi-modal data includes: Obtain the three-dimensional point cloud data, calculate the centroid of the three-dimensional point cloud to get the center point of the caisson ceiling, and obtain the symmetry axis of the point cloud through the principal component analysis method.

[0034] In a specific embodiment, the formula for obtaining the center point of the caisson by calculating the centroid of the three-dimensional point cloud is as follows: where C represents the coordinates of the calculated geometric center point; N represents the total number of points in the point cloud data; x i , y i , z i respectively represent the x, y, and z coordinate values of the i-th point in the three-dimensional space.

[0035] In addition, the position of the center point can be further adjusted by the Iterative Closest Point (ICP) algorithm or other optimization methods to make it more consistent with the actual geometric center of the caisson: where C opt represents the coordinates of the center point obtained after optimization, ICP is the Iterative Closest Point algorithm for optimizing the position of the center point, and L1” represents the point cloud data after preprocessing (such as denoising, downsampling, etc.).

[0036] In a specific embodiment, the specific steps for obtaining the symmetry axis of the point cloud by the principal component analysis method are as follows: Principal Component Analysis (PCA): By performing principal component analysis on the point cloud data, the main direction of the point cloud data, that is, the direction of the symmetry axis, is found.

[0037] First, calculate the covariance matrix: where P L represents the coordinate vector of the i-th point in the point cloud.

[0038] Calculate the eigenvalues and eigenvectors: represents the j-th eigenvector of the matrix Σ, which is a column vector; λj represents the j-th eigenvalue of the matrix Σ; Select the eigenvector corresponding to the largest eigenvalue as the direction of the symmetry axis: where, represents the eigenvector with the largest eigenvalue.

[0039] Combined with the optimized center point C opt and the direction vector , it represents the symmetry axis A.

[0040] where t is a real number parameter.

[0041] In a specific embodiment, the hierarchical boundaries of the caisson ceiling (such as the bracket layer and the well center layer) are obtained using point cloud data. Geometric analysis and image processing techniques can be used to extract features from the point cloud data. For example, edge detection algorithms (such as the Canny edge detector) are applied to identify the edges of the structure, and these edges may correspond to the boundaries of different levels; methods such as the RANSAC algorithm are used to fit planes to identify possible hierarchical structures. For example, the bracket layer and the well center layer may appear as different planes.

[0042] In a specific embodiment, since the caisson ceiling is at the top of building components and it is not convenient to manually add positioning points, a laser emission source is needed here to locate the information of the key positions of the caisson ceiling, and basic point cloud data such as the center point, symmetry line, and edge information of the caisson ceiling are extracted. After this information is collected, it will be transmitted to the control center (software) through a wireless transmission module. For the key position points of the caisson ceiling, a lidar is used. First, the target points that need to be acquired are manually determined, such as the center point, symmetry line, edge position, etc., and then the point cloud data of these key points are acquired through the lidar, and then transmitted to the control center through the wireless transmission module.

[0043] The key target points here mainly include: 1) Center point: It is used to confirm the position center of the caisson ceiling. If the building is displaced and the position of the caisson ceiling changes, it can be identified through the center point; 2) Symmetry line: The horizontal and vertical symmetry lines of the caisson ceiling are collected, so that problems can be found when the caisson ceiling itself has an asymmetric deformation; 3) Edge position: These data are mainly the border positions of the caisson ceiling. If the border of the caisson ceiling becomes loose, the abnormality can be judged through the edge position at this time; 4) Mortise and tenon joints: It is mainly the position positioning of some key mortise and tenon components. If there are problems such as damage or looseness of the mortise and tenon, it can be judged through the changes of the mortise and tenon joints; 5) Pattern turning points: It is mainly to identify the positions of some key patterns. Through these data, the relative positions of each key component can be judged. If there is a deformation of the pattern, the abnormality can be judged through these relative positions.

[0044] Step 3: Align the three-dimensional point cloud data, multi-spectral image data, and macro image data using the center point and symmetry axis of the caisson ceiling.

[0045] In a specific embodiment, aligning the three-dimensional point cloud data, multi-spectral image data, and macro image data using the center point and symmetry axis of the caisson ceiling includes: Feature extraction is respectively performed on the three-dimensional point cloud data, multi-spectral image data, and macro image data to obtain the key feature points and feature vectors of the multi-modal data; Calculate the distances between the feature vectors of the three-dimensional point cloud data, multi-spectral image data, and macro image data, and use the nearest neighbor search to filter out the matching pairs.

[0046] Taking the center point of the caisson as the reference, use the symmetry axis to align the matching key feature points.

[0047] In a specific embodiment, during the central symmetry constraint alignment process, the algorithm may search for key feature points (such as mortise and tenon joints, pattern turning points) and try to find their corresponding points in different data sources. By matching these feature points, the algorithm can determine the relative positions and orientations between different data sources, thereby achieving more accurate alignment. This method is particularly suitable for processing ancient architectural elements with repetitive patterns or structural symmetries, such as the caisson in this application.

[0048] Step Four: Perform hierarchical data fusion on the three-dimensional point cloud data, multi-spectral image data, and macro image data to obtain the fused caisson model to be repaired.

[0049] In a specific embodiment, performing hierarchical data fusion on the three-dimensional point cloud data, multi-spectral image data, and macro image data to obtain the fused caisson model to be repaired includes: Extract the geometric structure features of the caisson from the three-dimensional point cloud data; extract the texture distribution features, pattern turning points, and spectral material attribute data of the painted layer from the multi-spectral image data; extract the microscopic detail features from the macro image data; Use the geometric structure features to construct a three-dimensional model of the caisson structure layer; combine the texture distribution features and pattern turning points in the multi-spectral image data with the microscopic detail features in the macro image data to construct a texture model of the painted layer; align the microscopic detail features and spectral material attribute data into the three-dimensional model to generate a labeled material layer.

[0050] Step Five: Extract the missing pattern data from the caisson model to be repaired, input the missing pattern data into a pre-trained domain adaptation generative adversarial network to generate repair suggestions, and compare with the knowledge graph in real time to constrain the repair suggestions. Refer to Figure 3 .

[0051] In a specific embodiment, traditional methods are prone to ignoring the hierarchical regulations of ancient architectural paintings (such as "hexi color painting" is only used for imperial buildings), resulting in the repair results violating historical authenticity. In this application, by inputting the incomplete pattern area ( Figure 3 in step S05), generate repair suggestions through a pre-trained domain adaptation generative adversarial network (DA-GAN), and compare with the knowledge graph in real time for constraint. Access the knowledge base of Ming and Qing official architectural paintings to verify the pattern style of the repair plan (such as the scale density of "gold dragon hexi" and the construction logic of dougong) ( Figure 3 in step S06).

[0052] Pattern knowledge graph: For example, encoding the rules of Ming and Qing official architectural paintings (such as pattern types, color combinations, composition ratios) into a structured knowledge base. For example, in the "Golden Dragon and Imperial Seal" painting, the scale density of the dragon pattern is 8 - 12 scales / cm, and the head orientation needs to conform to the "centripetal" principle. The completion of the dougong pattern needs to follow the construction logic of "single overhang and single cantilever with five steps".

[0053] In another specific embodiment, a pre - constructed knowledge graph is used to train a domain - adversarial generative network (DA - GAN). When training the DA - GAN, the loss function usually includes two parts: one part is the loss of the traditional generative adversarial network (GAN), and the other part is the domain - adversarial loss, which uses the knowledge graph to enhance the discriminator's ability to distinguish between the source domain and the target domain: 1. Traditional GAN loss: The discriminator D tries to distinguish between real samples and generated samples, and its loss function can be expressed as: where, denotes the expectation of the sample x under the real data distribution ; logD(x) represents the logarithmic probability that the discriminator D judges the sample x as a real sample; denotes the expectation of the noise z under the prior distribution ; G(z) represents the sample generated by the generator G according to the noise z. denotes the prior distribution of the noise input to the generator, usually a uniform distribution or a Gaussian distribution.

[0054] 2. Domain - adversarial loss: The domain - adversarial loss aims to make the discriminator unable to distinguish whether the sample comes from the source domain or the target domain. This can be achieved by maximizing the domain classification loss of the source - domain and target - domain samples on the discriminator output: where, denotes the expectation of the sample x under the source - domain sample distribution ; logD domain (x) represents the logarithmic probability that the domain - classification part D domain of the discriminator D judges the source - domain sample x as belonging to the source domain. denotes the expectation of the sample under the target - domain sample distribution ; represents the logarithmic probability that the domain - classification part D domain of the discriminator D judges the target - domain sample as not belonging to the source domain (i.e., belonging to the target domain).

[0055] 3. Knowledge Graph Enhanced Loss: Utilize the structural information in the knowledge graph to enhance the domain adversarial loss. This can be achieved by maximizing the distance between the embeddings of source domain and target domain entities in the knowledge graph: where, indicates that entity e belongs to the union of the source domain entity set E src and the target domain entity set E tgt . D kg (e) represents the embedding representation of entity e by the knowledge graph embedding part of discriminator D. T(e) represents the corresponding entity of entity e in the target domain. Here, it is assumed that there exists a mapping T that can map entities in the source domain to their corresponding entities in the target domain.

[0056] 4. Total Loss Function: Combine the above three parts of losses to obtain the total loss function of the discriminator: where, λ adv and λ kg are weight parameters used to balance the contributions of different loss terms.

[0057] Based on the AI repair method driven by cultural context, construct a caisson pattern knowledge graph (incorporating the regulations of Ming and Qing official architectural color paintings), train a domain adaptive generative adversarial network (DA-GAN), and when complementing incomplete patterns, automatically follow the historical style (such as the scale density and orientation of the "Golden Dragon and Imperial Seal" pattern) to avoid "style conflicts".

[0058] Step Six: Repair the caisson using the constrained repair suggestions.

[0059] In a specific embodiment, Step Six further includes: Obtain the repaired caisson model, generate incremental changes by comparing the differences between the repaired model and the original model point by point, and use a visualization tool to label the incremental changes on the repaired model.

[0060] For example, each time a new version is generated after repair (such as B2 in step S07 of Figure 3 ), the system automatically labels the incremental changes (such as highlighting the repaired area in green), supporting users to compare historical versions (B1 vs B2).

[0061] In a specific embodiment, Step Six further includes: Export the repaired caisson model in a format supported by AR devices; Import the exported model into the software platform of AR devices; Project the incremental changes onto the caisson ceiling body through the AR device, and perform model repair through the interaction of gesture operations with the AR model.

[0062] Through this AR interaction display method, import the fusion model into the AR device (such as Hololens), and the user can gesture-operate to "disassemble" the caisson ceiling structure layer to view the repair effect.

[0063] In a specific embodiment, for example: 1. Data collection: Laser scanning found that there is a 0.2 mm crack (L1) in the mortise and tenon joint, and multispectral imaging detected that the painted brackets faded (M1).

[0064] 2. Adaptive alignment: The system automatically aligns the crack and the faded area based on the symmetry axis to locate the repair priority.

[0065] 3. Intelligent repair: DA-GAN generates a repair plan for the painted bracket pattern, and the knowledge graph verifies that it conforms to the "single cantilever and single angled, five-step" regulation.

[0066] 4. AR verification: The repair plan is projected onto the caisson ceiling body through AR, and after the cultural relics restorer confirms it is correct, it is executed.

[0067] In a specific embodiment, the method further includes step seven: obtaining real-time monitoring data in the caisson ceiling, using the real-time monitoring data to predict the deformation risk of the caisson ceiling and push a reinforcement plan.

[0068] In a specific embodiment, obtaining real-time monitoring data in the caisson ceiling, using the real-time monitoring data to predict the deformation risk of the caisson ceiling and push a reinforcement plan includes: Regularly collect environmental data and structural deformation data based on real-time monitoring devices; Transmit the collected data to the central control system through a wireless network; Input the structural deformation data and real-time environmental data into the LSTM time series analysis model to predict the deformation trend, evaluate the deformation risk level according to the predicted deformation trend, and generate a reinforcement plan according to the deformation risk level.

[0069] In a specific embodiment, using this dynamic risk warning mechanism, develop a deformation-environment coupling analysis model, predict the deformation trend of the caisson ceiling through time series analysis (such as the LSTM network), and locate high-risk areas (such as the fatigue threshold of the mortise and tenon joints in the northwest corner under humidity cycling).

[0070] In a specific embodiment, for example, during the acquisition of multi - spectral images, if deformation is detected outside the caisson ceiling through edge detection and deformation monitoring algorithms, different alarm messages will be given according to the degree of deformation; if during the acquisition of macro - images, risks such as possible patterns and cracks are identified through deep learning, the risk types and levels of specific patterns will be given.

[0071] In a specific embodiment, if the meteorological environment deteriorates, such as in extreme weather conditions like long - term exposure to a humid environment, continuous high temperature, or strong wind, damage detection and early warning will be automatically triggered.

[0072] In a specific embodiment, the influencing factors of caisson ceiling deformation mainly include: 1) External climate factors, including internal and external temperature and humidity, light, wind and sand, etc., which will affect the wooden structure, painted decoration, etc. of the caisson ceiling body.

[0073] 2) Structural changes in the caisson ceiling itself caused by factors such as structural fatigue and physical factors.

[0074] Therefore, for the acquisition of monitoring data of the caisson ceiling, it includes the acquisition of external environmental data and internal caisson ceiling body data.

[0075] Acquisition of external climate information: The acquisition of external climate factors is mainly achieved by sensors such as temperature and humidity sensors, light sensors, and wind sensors to collect meteorological information. These sensors are distributed inside and outside the building of the caisson ceiling, and are deployed at multiple points to collect meteorological factors at different heights and positions.

[0076] Acquisition of caisson ceiling body information: The acquisition of caisson ceiling body information mainly relies on cameras. Multiple cameras will be set inside the building to collect image information of key parts of the caisson ceiling respectively.

[0077] After the above - mentioned external climate information and caisson ceiling body information are acquired, they will be transmitted to a unified controller through network devices for secondary processing.

[0078] In a specific embodiment, embedded flexible sensors are set in the caisson ceiling. For example, flexible strain sensors are deployed at hidden places of the caisson ceiling (such as mortise - and - tenon joints), which can bend with the deformation of the wood to achieve non - invasive monitoring (traditional rigid sensors need to be installed by drilling, which will damage the cultural relics body).

[0079] In a specific embodiment, step seven also includes: using real - time monitoring equipment to collect meteorological data and caisson ceiling state data of the environment where the caisson ceiling is located, calculating meteorological disturbance factors using the meteorological data and caisson ceiling state data, and correcting the deformation risk level using the meteorological disturbance factors.

[0080] In a specific embodiment, after collecting the corresponding meteorological information, it is necessary to construct a perturbation factor based on some current meteorological information. That is, when judging the microscopic deformation of the external contour or pattern of the caisson ceiling, the influence of current climate factors needs to be considered. If the current temperature is too high, the influence of thermal expansion and contraction of the wooden structure needs to be considered, and the corresponding perturbation coefficient in the fault identification process is adjusted at this time. If the wind force is large, the perturbation coefficient in the model is adjusted according to the specific wind force level.

[0081] Through a differentiated innovation path, this application innovatively proposes a multi-modal data fusion modeling - presenting a heterogeneous data alignment algorithm for lidar + multi-spectral imaging + macro photography; culture context-driven AI restoration - constructing a knowledge graph of caisson ceiling patterns (incorporating the regulations of colored paintings in Ming and Qing official architecture), and training a domain-adaptive generative adversarial network (DA-GAN); a dynamic risk warning mechanism - developing a deformation-environment coupling analysis model, predicting the deformation trend of the caisson ceiling through time series analysis (such as an LSTM network), and locating high-risk areas (such as the fatigue threshold of the mortise and tenon joints in the northwest corner under humidity cycling). Through the three major innovations of structure-driven alignment, culture-embedded restoration, and dynamic closed-loop update, this application breaks through the bottleneck of data fragmentation and style distortion in traditional digital protection, providing a full-chain solution from "salvage restoration" to "preventive governance" for wooden structure heritages such as caisson ceilings, with both academic value and industrial potential.

[0082] Further reference Figure 4 As an implementation of the above method, this application provides an embodiment of a caisson ceiling maintenance system. This device embodiment corresponds to Figure 1 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0083] Reference Figure 4 A caisson ceiling maintenance system includes: A multi-modal data acquisition module 101 configured to acquire multi-modal data of the caisson ceiling; the multi-modal data includes three-dimensional point cloud data, multi-spectral image data, and macro image data; A geometric calculation module 102 configured to calculate the center point and symmetry axis of the caisson ceiling using the multi-modal data; A data alignment module 103 configured to align the three-dimensional point cloud data, multi-spectral image data, and macro image data using the center point and symmetry axis of the caisson ceiling; The hierarchical fusion module 104 is configured to perform hierarchical data fusion on the 3D point cloud data, multispectral image data, and macro image data to obtain a fused algae well model to be repaired. Specifically, the hierarchical fusion module is specifically configured to extract the geometric structure features of the algae well from the 3D point cloud data; extract the texture distribution features, pattern turning points, and spectral material attribute data of the painted layer from the multispectral image data; extract the microscopic detail features from the macro image data; use the geometric structure features to construct a 3D model of the algae well structure layer; combine the texture distribution features and pattern turning points in the multispectral image data with the microscopic detail features in the macro image data to construct a texture model of the painted layer; and align the microscopic detail features and spectral material attribute data into the 3D model to generate a labeled material layer. The missing pattern completion module 105 is configured to extract the missing pattern data from the algae well model to be repaired, input the missing pattern data into a pre-trained domain adaptation generative adversarial network to generate repair suggestions, and compare the knowledge graph in real time to constrain the repair suggestions. The repair module 106 is configured to repair the algae well using the constrained repair suggestions.

[0084] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0085] On the other hand, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist separately without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is caused to execute the method as shown in Figure 1 the above.

[0086] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element.

[0087] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a computer terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0088] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these are within the protection scope of the present invention.

Claims

1. A method for maintaining a caisson ceiling, characterized in that, Including the steps: Collect multi-modal data of the caisson ceiling; the multi-modal data includes 3D point cloud data, multi-spectral image data, and macro image data; Calculate the center point and the symmetry axis of the caisson ceiling using the multi-modal data; Align the 3D point cloud data, multi-spectral image data, and macro image data using the center point and the symmetry axis of the caisson ceiling; Perform hierarchical data fusion on the 3D point cloud data, multi-spectral image data, and macro image data to obtain a fused caisson ceiling model to be repaired; Extract the missing pattern data from the caisson ceiling model to be repaired, input the missing pattern data into a pre-trained domain adaptation generative adversarial network to generate repair suggestions, and compare with the knowledge graph in real time to constrain the repair suggestions; Repair the caisson ceiling using the constrained repair suggestions; Among them, performing hierarchical data fusion on the 3D point cloud data, multi-spectral image data, and macro image data to obtain a fused caisson ceiling model to be repaired includes: Extract the geometric structure features of the caisson ceiling from the 3D point cloud data; extract the texture distribution features, pattern turning points, and spectral material attribute data of the painted layer from the multi-spectral image data; extract microscopic detail features from the macro image data; Construct a 3D model of the caisson ceiling structure layer using the geometric structure features; combine the texture distribution features and pattern turning points in the multi-spectral image data with the microscopic detail features in the macro image data to construct a texture model of the painted layer; align the microscopic detail features and spectral material attribute data into the 3D model to generate a labeled material layer.

2. The algae well maintenance method according to claim 1, characterized in that, The calculating the center point and the symmetry axis of the caisson ceiling using the multi-modal data includes: Obtain the 3D point cloud data, calculate the centroid of the 3D point cloud to obtain the center point of the caisson ceiling, and obtain the symmetry axis of the point cloud through the principal component analysis method.

3. The method for maintaining a caisson ceiling according to claim 2, wherein , the aligning the 3D point cloud data, multi-spectral image data, and macro image data using the center point and the symmetry axis of the caisson ceiling includes: Extract features from the 3D point cloud data, multi-spectral image data, and macro image data respectively, and obtain the key feature points and feature vectors of the multi-modal data; Calculate the distances between the feature vectors of the 3D point cloud data, multi-spectral image data, and macro image data, and use the nearest neighbor search to screen the matching pairs; Taking the center point of the caisson ceiling as the reference, align the matching key feature points using the symmetry axis.

4. The method for maintaining a caisson ceiling according to claim 3, characterized in that, The calculation formula for calculating the centroid of the 3D point cloud to obtain the center point of the caisson ceiling is: ， Among them, C represents the coordinates of the calculated geometric center point; N represents the total number of points in the point cloud data; x i , y i , z i respectively represent the x, y, and z coordinate values of the i-th point in three-dimensional space.

5. A method for maintaining a caisson ceiling according to any one of claims 1-4, characterized in that, It also includes the steps: Obtain the repaired caisson ceiling model, generate an incremental change by comparing the differences between the repaired model and the original model point by point, and mark the incremental change on the repaired model using a visualization tool.

6. The method for maintaining a caisson ceiling according to claim 5, wherein It also includes the steps: Export the repaired caisson ceiling model into a format supported by the AR device; Import the exported model into the software platform of the AR device; Project the incremental change onto the caisson ceiling body through the AR device, and perform model repair through the interaction of gesture operations with the AR model.

7. A method for maintaining a caisson ceiling according to any one of claims 1-4, characterized in that It also includes the steps: Obtain the real-time monitoring data in the caisson ceiling, predict the deformation risk of the caisson ceiling using the real-time monitoring data, and push a reinforcement plan.

8. A method for maintaining a caisson ceiling according to claim 7, characterized in that, The obtaining of real-time monitoring data in the caisson ceiling, predicting the deformation risk of the caisson ceiling using the real-time monitoring data and pushing a reinforcement plan includes: Regularly collecting environmental data and structural deformation data based on real-time monitoring devices; Transmitting the collected data to a central control system through a wireless network; Inputting the structural deformation data and real-time environmental data into an LSTM time series analysis model to predict the deformation trend, evaluating the deformation risk level according to the predicted deformation trend, and generating a reinforcement plan according to the deformation risk level.

9. The method for maintaining a caisson ceiling according to claim 8, characterized in that, It further includes the steps of: Collecting meteorological data and caisson ceiling state data of the environment where the caisson ceiling is located using real-time monitoring devices, calculating a meteorological disturbance factor using the meteorological data and caisson ceiling state data, and correcting the deformation risk level using the meteorological disturbance factor.

10. An algae well maintenance system, characterized in that, It includes: A multi-modal data acquisition module configured to acquire multi-modal data of the caisson ceiling; the multi-modal data includes three-dimensional point cloud data, multi-spectral image data, and macro image data; A geometric calculation module configured to calculate the center point and symmetry axis of the caisson ceiling using the multi-modal data; A data alignment module configured to align the three-dimensional point cloud data, multi-spectral image data, and macro image data using the center point and symmetry axis of the caisson ceiling; A hierarchical fusion module configured to perform hierarchical data fusion on the three-dimensional point cloud data, multi-spectral image data, and macro image data to obtain a fused caisson ceiling model to be repaired; A missing pattern completion module configured to extract missing pattern data from the caisson ceiling model to be repaired, input the missing pattern data into a pre-trained domain adaptation generative adversarial network to generate repair suggestions, and compare with a knowledge graph in real time to constrain the repair suggestions; A repair module configured to repair the caisson ceiling using the constrained repair suggestions; Among them, the hierarchical fusion module is specifically configured to extract the geometric structure features of the caisson ceiling from the three-dimensional point cloud data; extract the texture distribution features, pattern turning points, and spectral material attribute data of the painted layer from the multi-spectral image data; extract microscopic detail features from the macro image data; Construct a three-dimensional model of the caisson ceiling structure layer using the geometric structure features; combine the texture distribution features and pattern turning points in the multi-spectral image data with the microscopic detail features in the macro image data to construct a texture model of the painted layer; align the microscopic detail features and spectral material attribute data into the three-dimensional model to generate a labeled material layer.

Images