Automatic monitoring system and method for cliff stone carving diseases

Abstract

The invention relates to a cliff stone carving disease automatic monitoring system, which automatically identifies a stone carving picture based on a deep learning model, identifies the identity of a specific stone carving and a surface disease thereof, on one hand, displays a detailed cultural background and historical value to tourists, and enhances the visiting experience of the tourists; on the other hand, disease data are continuously accumulated through stone carving pictures shot by tourists, and a dynamic monitoring data stream is formed. According to the method, the cliff stone carving disease identification result of the target detection model is combined with interactive feedback of tourists, regular analysis is performed, a disease development trend report is generated, convenient and efficient disease monitoring service is provided for the tourists and cultural relic protection personnel, and automatic, intelligent and visual disease monitoring and historical data analysis are realized.

Description

Technical Field

[0001] The invention relates to the technical field of cultural relics protection, and in particular to an automatic monitoring system and method for cliff carving diseases. Background Art

[0002] The protection and disease monitoring methods of cliff carvings mostly rely on traditional means, mainly by cultural relics protection personnel taking photos on site, and then manually observing the photos to identify and analyze the type of disease. This method is limited by personnel experience, has low recognition efficiency, is difficult to promote in large-scale stone carving protection, and cannot grasp the development dynamics of stone carving diseases in real time. Existing intelligent detection methods and equipment are mostly used for indoor and movable cultural relics (such as ceramics, calligraphy and painting cultural relics, etc.), while there is still a lack of dynamic monitoring solutions for large-scale open-air cultural relics (such as cliff carvings) diseases.

[0003] The invention patent with the publication (announcement) number CN118883730A discloses a method and device for monitoring the damage of cultural relics by using acoustic emission multi-source information. Relying on acoustic emission sensors and environmental sensors, the acoustic emission signals and environmental information of cultural relics in a specific environment are collected in real time. Through the pre-trained machine learning model, the system can determine the location of the damage of the cultural relics and further analyze the type of damage. Acoustic emission technology has advantages in detecting internal defects of materials, but the solution relies on sensing equipment and is mainly suitable for cultural relics in indoor or controllable environments. For large open-air cultural relics such as cliff carvings, it is difficult to achieve large-scale real-time disease monitoring.

[0004] The invention patent with the publication (announcement) number CN117853459A discloses an automatic high-precision extraction method for cultural relics diseases based on three-dimensional fine data. It mainly uses the grayscale co-occurrence matrix of two-dimensional images and three-dimensional reconstruction technology to automatically obtain the fine geometric information of cultural relics diseases. Through algorithms such as SLIC image segmentation and superpixel clustering of K values, the system can extract the spatial coordinates of cultural relics diseases and provide high-precision disease analysis results through three-dimensional reconstruction technology. This method is more efficient for the extraction of precise cultural relics diseases, but its premise is to obtain high-quality images and accurate three-dimensional data, which is difficult to meet in outdoor environments, especially in cliff carvings.

[0005] The invention patent with the publication (announcement) number CN116818680A discloses an automatic detection method for cultural relics diseases. It synthesizes the three-dimensional image of the cultural relics through photogrammetry and generates six-view images of the disease in the light source scene. Using the model background color and color difference scanning technology, the system can determine the specific location and type of disease. This technical method requires specific light source and material background color control, which is suitable for cultural relics management in museums or warehouses, but is difficult to apply to stone carvings in the outdoors and exposed to the natural environment.

[0006] The invention patent with the publication (announcement) number CN115049915A discloses a method for identifying calligraphy and painting cultural relics defects based on target detection technology. It adopts target detection technology and uses deep learning algorithms to identify calligraphy and painting cultural relics defects. Through self-adversarial training, CmBN strategy and other optimization models, the surface defects of cultural relics are detected and located, which is suitable for the detection of calligraphy and painting cultural relics on two-dimensional planes. However, since this method is mainly based on the disease characteristics of calligraphy and painting cultural relics, it faces certain challenges when directly applied to three-dimensional irregular surfaces (such as cliff carvings), and it has not been applied to the disease monitoring of large outdoor cultural relics such as stone carvings.

[0007] It can be seen that although the existing technologies have achieved certain results in disease identification and three-dimensional data reconstruction, their application environment and technical limitations make it difficult to realize automation and intelligence in the monitoring of large-scale, open-air cliff carving diseases. First, most existing technologies require a fixed monitoring environment and precision equipment, which limits their application in outdoor open cultural relics sites and makes it difficult to meet the real-time and convenient monitoring needs of outdoor cultural relics such as cliff carvings. Secondly, although traditional three-dimensional modeling and superpixel segmentation methods can provide high-precision data, their complex operating procedures and expensive hardware dependence increase monitoring costs and time, limiting their large-scale application. In addition, disease extraction based on image chromatic aberration is unstable in outdoor environments with complex lighting and varying natural conditions. The deep learning target detection method relies heavily on data annotation, has limited recognition accuracy and wide applicability, and fails to use disease recognition results for further continuous monitoring or prediction. Summary of the invention

[0008] The purpose of the present invention is to provide an automatic monitoring system for cliff carving defects, which automatically identifies stone carvings and defect information by taking photos and uploading them, combines deep learning models, target detection and defect identification, and generates data support to achieve long-term monitoring of defect changes, form continuous updating of defect data, realize real-time monitoring of defects and data accumulation at low cost, and promote efficient protection and management of cultural heritage.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] The invention provides an automatic monitoring system for cliff carving diseases, comprising a tourist end, a server, a target detection model and a management end; at the tourist end, tourists take photos of the stone carvings and upload them to the server, and the server automatically identifies the photos and returns the historical background and disease status of the stone carvings; the server receives the photos of the cliff carvings uploaded by the tourists and transmits them to the target detection model, and stores the identification results and updates the database; the target detection model performs stone carving and disease detection based on an improved Faster R-CNN model, and identifies the disease type and location; cultural relics protection personnel view real-time identification data through the management end, including disease images of the cliff carvings and distribution statistics of disease types, as well as long-term data analysis.

[0011] The present invention also provides a method for automatically monitoring cliff carving defects, using the automatic monitoring system for cliff carving defects as claimed in claim 1, characterized in that it comprises the following steps:

[0012] Step 1: The tourist takes a clear picture of the cliff carvings and uploads it to the server;

[0013] Step 2: After receiving the uploaded image, the server pre-processes the image;

[0014] Step 3: Optimize and improve the existing Faster R-CNN model to adapt it to the unique characteristics of the cliff carvings, which have complex textures and minute disease details; the target detection model includes the improved Faster R-CNN model; the improved Faster R-CNN model is trained to identify the types of diseases on the cliff carvings; the preprocessed image is input into the improved Faster R-CNN model for identification, and the diseased area is located by combining the region proposal network and the feature pyramid network, and the disease type, diseased area and disease time are formed into disease information;

[0015] Step 4: The server receives and stores the disease information in the database;

[0016] Step 5: Use data analysis algorithms to conduct statistical and predictive modeling of disease types and changing trends;

[0017] Step 6: The server pushes and displays the changing trend and statistical distribution of cliff carving diseases to the management end.

[0018] In the above-mentioned automatic monitoring method for cliff carving defects, the method of locating defect areas using a region proposal network and a feature pyramid network includes the following steps:

[0019] Step 1: Introduce the feature pyramid network and use multi-layer feature fusion to generate a multi-scale pyramid feature map. l The fusion process is shown in formula (1):

[0020] P l =Conv(C l )+Upsample(P l+1 ) (1)

[0021] Where:

[0022] Conv(C l ) represents the convolution feature map C l Perform 1×1 convolution operation to unify the number of channels; the output feature map of the convolution layer can be represented by C2, C3, C4, and C5, where C2 is the shallow feature map with the highest resolution and C5 is the deep feature map with the strongest semantic information;

[0023] Upsample(P l+1 ) represents the feature map P of the previous pyramid layer l+1 Upsample to the same resolution as C l Matching; Through the top-down feature fusion path, the high-semantic deep features are gradually combined with the high-resolution shallow features to generate the output feature maps P2, P3, P4, P5 of each layer of the feature pyramid;

[0024] Each pyramid level feature P l The local feature map is further extracted through 3×3 convolution for subsequent target detection;

[0025] Step 2: First, run the region proposal network on each layer of the feature pyramid network to generate multi-scale candidate region proposal boxes (Anchors); calculate the coverage target probability p and boundary regression offset t of each candidate region for each Anchor according to formula (2) x ,t y ,t w ,t h :

[0026] {p,t x ,t y ,t w ,t h =RPN(P l ) (2)

[0027] In the formula, RPN(P l ) represents the feature map P at each layer of the feature pyramid network l Run the region proposal network on it to generate multi-scale candidate region proposal boxes; Anchor covers a variety of disease targets from small cracks to large-scale peeling; l represents the index of a layer in the FPN feature pyramid;

[0028] The anchor is then mapped back to the image coordinates, and the box with the highest confidence is selected from the overlapping candidate region proposal boxes according to the preset non-maximum suppression (NMS) algorithm;

[0029] Step 3: Feature fusion and detection: Through the feature map provided by the feature pyramid network, regional features are extracted from the feature map that best suits the target scale of the disease. For small cracks, the high-resolution feature maps of P2 and P3 are used to enhance edge information. For color difference diseases, the deep semantic features and shallow detail features are combined to identify color differences. The final disease type classification and disease area boundary regression are optimized through formulas (3), (4), (5), and (6):

[0030] Classification of losses by disease type L cls Evaluate the matching degree between the class probability predicted by the model and the true label, calculated by formula (3):

[0031]

[0032] where y i is the true category label, is the predicted category probability;

[0033] The bounding box regression error is calculated by smoothing the L1 loss to avoid excessive punishment for large errors; the boundary regression loss L reg Calculated by formula (4):

[0034]

[0035] In the formula, t i is the boundary value of the real box (center point and size); is the boundary value of the prediction box.

[0036] The loss function is smooth L1 (x) is calculated by formula (5):

[0037]

[0038] In the formula, x represents the difference (error) between the target value and the predicted value, which is calculated as follows: Where: t is the target value, such as the coordinate value of the actual position of the bounding box; is the predicted value, such as the bounding box position predicted by the model; |x|: represents the absolute value of the error;

[0039] The total loss function L is calculated by formula (6):

[0040] L=L cls +λL reg (6)

[0041] Where λ is the weight factor for adjusting classification loss and regression loss;

[0042] The total loss function L comprehensively weighs the classification accuracy and bounding box positioning accuracy.

[0043] In the above-mentioned automatic monitoring method for cliff carving defects, the preprocessing operation in step 2 includes one or more operations of image cropping, noise removal, contrast enhancement, image color correction, edge enhancement, gamma correction, distortion correction, and image resolution improvement.

[0044] In the above-mentioned automatic monitoring method for cliff carving diseases, the disease types in step 3 include one or more of peeling, cracks, weathering, fissures, dust accumulation, water rust crusting, local loss, surface dissolution, powdering, lamellar peeling, alkali and salt expansion, and low-level plant damage.

[0045] In the above-mentioned automatic monitoring method for cliff carving diseases, in step 5, the data analysis algorithm is time series analysis and regression model; the MySQL database is used to store cliff carving images and disease information, and the Python data analysis library is used to perform statistical analysis and predictive modeling of the disease information data.

[0046] In the above-mentioned automatic monitoring method for cliff carving defects, in step 3, the Faster R-CNN model used to identify the type of cliff carving defects is replaced by a YOLO model or a method of fusing image recognition with text and 3D point cloud multimodal data.

[0047] In the above-mentioned automatic monitoring method for cliff carving defects, in step 3, the Faster R-CNN model used to identify the type of cliff carving defects is replaced by a U-Net semantic segmentation model or a DeepLab semantic segmentation model.

[0048] In the above-mentioned automatic monitoring method for cliff carving defects, step 3 also includes an expert system; the expert system is used to automatically judge the defects of the cliff carvings.

[0049] The beneficial effects of the present invention are:

[0050] The present invention is suitable for efficient and convenient disease monitoring of outdoor cultural relics such as cliff carvings. It can automatically identify the stone carvings and disease information by taking photos and uploading them, and generate data support to achieve long-term monitoring of disease changes. It solves the shortcomings of existing technologies in outdoor scene adaptability, convenience and automated monitoring. It guides tourists to participate through an open platform, uses tourists' photos to continuously update disease data, and achieves real-time monitoring of diseases and data accumulation at low cost, promoting efficient protection and management of cultural heritage.

[0051] The present invention combines deep learning models, target detection and systematic monitoring of disease identification, realizes stone carving identification and disease monitoring through tourists' photography, accumulates disease data, and provides a more comprehensive and timely monitoring method for stone carving protection, making up for the shortcomings of existing technologies in large-scale outdoor cultural relics disease monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the system structure principle of the present invention;

[0053] Figure 2 This is a schematic diagram of the visitor terminal identification process of the present invention;

[0054] Figure 3 This is a schematic diagram of the disease data monitoring process of the management end of the present invention;

[0055] Figure 4 This is a screenshot of the recognition effect of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0057] refer to Figures 1 to 4 The present invention provides a cultural relic protection system for disease identification and monitoring based on tourists taking photos and uploading stone carving pictures. The target detection model is used to identify the stone carving photos, which can not only provide tourists with cultural and historical background information of the stone carvings, but also automatically identify and continuously monitor the diseases of cultural relics through the continuously uploaded photos. The present invention is divided into four modules: data acquisition, disease identification, data analysis and display, and includes two parts: the tourist end and the management end.

[0058] In the stone carving disease detection and monitoring method of the present invention, the overall system process consists of multiple steps. Each step plays an important role in realizing the functions of automatic disease identification and long-term monitoring. The specific steps and processes are described as follows:

[0059] Step 1. Image acquisition and upload

[0060] Conditions: Visitors can use their mobile phones to take clear images of the stone carvings in the scenic area. The images must include the entirety of the stone carvings and the shooting environment must be well lit to clearly show the details of the stone carvings.

[0061] Function: This step provides the image data source for the entire system. The quality of image acquisition directly affects the accuracy and reliability of subsequent disease identification.

[0062] Step 2. Image preprocessing

[0063] Conditions: After receiving the images uploaded by visitors, the server automatically performs a series of preprocessing operations, including image cropping, noise removal, contrast enhancement, etc., to make the images clearer and meet the requirements of disease detection.

[0064] In addition, preprocessing operations include not only image cropping, noise removal, and contrast enhancement, but also a variety of operations that help improve model recognition accuracy, such as:

[0065] Image color correction: adjust the color distortion of the captured image to ensure the true restoration of the surface features of the stone carvings;

[0066] Edge enhancement: Enhance the contrast between the diseased area and the background by improving the prominence of the edges in the image;

[0067] Gamma correction: adjust image brightness to adapt to complex lighting environments;

[0068] Distortion correction: Corrects geometric deformation caused by shooting angle or equipment;

[0069] Image resolution enhancement: Improve image details through super-resolution reconstruction methods;

[0070] Remove high-frequency noise: Use frequency domain filtering technology to reduce high-frequency noise in the image that may interfere with recognition.

[0071] Function: Preprocessing can improve the accuracy of the detection model in extracting image features and ensure that subtle disease features are not covered by background noise. Through the above preprocessing operations, the image clarity and detail retention can be significantly enhanced, and the model's ability to extract and identify disease features can be improved. The preprocessed image is the input for subsequent detection links.

[0072] Relationship: Image preprocessing is a prerequisite for the operation of the detection model. Image quality is optimized through image enhancement and denoising, making the disease features more prominent and facilitating feature extraction by the detection model.

[0073] Step 3. Disease detection

[0074] Conditions: The preprocessed images are input into the improved Faster R-CNN model. The model is required to be trained extensively and be able to identify common types of defects (such as peeling, cracks, weathering, etc.) on stone carvings and locate the defect areas. Based on the Faster R-CNN model, the defect feature data is enhanced, and the region proposal network (RPN) and feature pyramid network (FPN) are combined to optimize the defect location accuracy and improve the recognition effect of small cracks and color difference defects.

[0075] In this paper, the Faster R-CNN model is optimized by introducing the feature pyramid network (FPN) to further improve the detection accuracy and positioning effect of cliff stone carving defects (such as small cracks and color difference defects). FPN significantly enhances the model's perception of multi-scale defect targets through a multi-scale feature fusion mechanism, and combines the region proposal network (RPN) and feature extraction network to achieve accurate identification of defects.

[0076] The core of the Feature Pyramid Network (FPN) is to generate multi-scale pyramid feature maps using multi-layer feature fusion. In the basic network (such as ResNet), the output feature map of the convolution layer can be represented by C2, C3, C4, and C5, where C2 is the shallow feature map with the highest resolution and C5 is the deep feature map with the strongest semantic information. Through the top-down feature fusion path, the high-semantic deep features are gradually combined with the high-resolution shallow features to generate the output feature maps p2, p3, p4, and p5 of each layer of the feature pyramid. The feature fusion process formula is as follows:

[0077] p l =Conv(C l )+Upsample(P l+1 ) (1)

[0078] in:

[0079] Conv(C l ) represents the convolution feature map C l Perform 1×1 convolution operation to unify the number of channels;

[0080] Upsample(P l+1 ) represents the feature map P of the previous pyramid layer l+1 Upsample to the same resolution as C l match.

[0081] Finally, each pyramid level feature P l Local features are further extracted through 3×3 convolution for subsequent target detection tasks.

[0082] Run the region proposal network (RPN) on each layer of the FPN feature map to generate multi-scale candidate region proposal boxes (Anchors). By setting different sizes of Anchors (such as 4x4 to 256x256 pixels) for each layer of the feature map, a variety of disease targets from small cracks to large-scale spalling can be covered. The specific calculation process is as follows:

[0083] 1. Calculate the target probability p and boundary regression offset t of the candidate box for each anchor x ,t y ,t w ,t h:

[0084] {p,t x ,t y ,t w ,t h =RPN(P l ) (2)

[0085] 2. Map the candidate box back to the image coordinates and filter out the high-confidence area according to the preset non-maximum suppression (NMS) algorithm.

[0086] In formula (2), RPN(P l ) represents the region proposal network (RPN) in a specific feature map P l The process of running on the network to generate candidate region proposal boxes (Anchors).

[0087] l: represents the index of a certain layer in the FPN feature pyramid (for example, from high-resolution shallow features to low-resolution deep features). Each layer of feature maps corresponds to a different scale and is used to capture a variety of disease targets (such as small cracks, large-area peeling, etc.).

[0088] Anchor is a set of candidate boxes of fixed size and aspect ratio generated by the region proposal network, which is used to preliminarily locate the target area to cover diseased areas of different sizes. Anchor size can range from small (such as 4×4 pixels) to large (such as 256×256 pixels).

[0089] The target probability p of the candidate box represents the probability of whether each anchor contains the target disease area, which is output by the classification branch of the region proposal network.

[0090] Boundary regression offset t x ,t y ,t w ,t h It is the displacement correction value between the predicted candidate box and the target box, which is output by the regression branch of the region proposal network.

[0091] t x ,t y is the offset of the center point coordinates; t w ,t h Offset for width and height.

[0092] The non-maximum suppression (NMS) algorithm is used to filter out the boxes with the highest confidence from the overlapping candidate boxes and eliminate redundancy, thereby retaining the optimal candidate area and reducing redundant detection results.

[0093] Feature fusion and detection: Through the pyramid feature map provided by FPN, RoI Align extracts regional features from the feature map that best suits the scale of the target disease to ensure high detection accuracy. For small cracks, the high-resolution feature map of P2 and P3 is used to enhance edge information; for color difference diseases, the deep semantic features and shallow detail features are combined to improve the ability to recognize color differences.

[0094] The final classification and boundary regression of the model are optimized by the following formula:

[0095] 1. Classification loss L cls (Cross entropy loss) is used to evaluate how well the class probabilities predicted by the model match the true labels:

[0096]

[0097] where y i is the true category label (such as disease type: crack, peeling, etc.), is the predicted category probability.

[0098] 2. Boundary regression loss L reg (Smooth L1 loss) The regression error of the bounding box is calculated by smoothing the L1 loss to avoid excessive punishment for large errors:

[0099]

[0100] In the formula, t i : The boundary value of the real box (center point and size), The boundary value of the prediction box.

[0101] in:

[0102]

[0103] It is a commonly used regression loss function, especially suitable for dealing with bounding box regression problems. It combines the advantages of L1 loss and L2 loss, and can be expressed as L2 loss (more sensitive to small errors) when the error is small, and as L1 loss (more robust to large errors) when the error is large.

[0104] x: represents the difference (error) between the target value and the predicted value, which is calculated as follows: in:

[0105] t: target value, such as the coordinate value of the true position of the bounding box.

[0106] Predicted values, such as the bounding box locations predicted by the model.

[0107] |x|: represents the absolute value of the error.

[0108] When |x|<1 (small error), the loss is 0.5x 2 (L2 loss form), the square form of the error can amplify the impact of small errors, thereby adjusting the model faster and refining the prediction.

[0109] When |x|≥1 (large error), the loss is |x|-0.5 (L1 loss form). The absolute error form avoids the excessive impact of large errors on the loss value, making the training process more stable.

[0110] At the position where the error |x|=1, the L2 loss and L1 loss are seamlessly connected:

[0111] 0.5(1 2 )=|1|-0.5=0.5

[0112] The function of Smooth L1 function:

[0113] (1) Sensitivity under small errors: When the error is small, Smooth L1 provides a square penalty to help the model fit the target value more precisely.

[0114] (2) Robustness under large errors: When the error is large, using L1 loss can avoid the impact of extreme errors, making the training process more stable.

[0115] The total loss function comprehensively weighs the classification accuracy and bounding box positioning accuracy, L is:

[0116] L=L cls +λL reg (6)

[0117] In the formula, λ is the weight factor for adjusting classification loss and regression loss.

[0118] Optimization effect analysis:

[0119] Small crack detection: Through the high-resolution features of the shallow feature map, the edge detail information is enhanced, significantly improving the model's sensitivity to small cracks.

[0120] Chromatic aberration disease detection: Combining deep semantic information and shallow detail information, the model’s ability to distinguish low-contrast diseases is enhanced through multi-scale fusion.

[0121] Multi-scale detection capability: The pyramid feature map of FPN effectively improves the model's detection capability for different disease targets (from tiny cracks to large-scale diseases).

[0122] Through these improvements, the model has achieved higher accuracy and robustness in the automatic detection of cliff carving defects, providing strong technical support for cultural relics protection.

[0123] It should be noted that the types of damage to cliff carvings are not limited to peeling, cracks and weathering, but are based on the classification system of the relevant national standards "Classification and Illustration of Damage to Stone Cultural Relics" (WW / T 0002-2007) and "Classification and Illustration of Damage to Brick and Stone Cultural Relics in Museums" (GB / T 30688-2014), specifically including:

[0124] Cracks: Cracks on the surface of stone carvings caused by stress or natural weathering;

[0125] Dust accumulation: dust and particles attached to the surface of stone carvings;

[0126] Water rust crust: A hardened layer formed by water erosion and mineral precipitation;

[0127] Partial loss: Part of the stone carving surface has fallen off due to weathering or human damage;

[0128] Surface dissolution: destruction of surface structure caused by chemical reactions;

[0129] Powdering: The particles on the surface of the stone carving become loose and peel off;

[0130] Flaking: The phenomenon of the surface of the stone carving peeling off in layers;

[0131] Salinity and salt bloom: surface diseases caused by salt deposition or dissolution;

[0132] Damage to lower plants: biological diseases caused by the attachment of plants such as mosses and algae.

[0133] Through accurate classification and identification of the above-mentioned types of diseases, not only can the common diseases of stone carvings be comprehensively covered, but also refined data support and prediction of disease development trends can be provided for cultural relics protection personnel.

[0134] The improved Faster R-CNN optimizes the existing Faster R-CNN model to adapt it to the unique characteristics of cliff carvings (such as complex textures and tiny damage details), thereby improving the accuracy and efficiency of recognition. The model can identify common types of cliff carving damage (such as peeling, cracks, weathering, etc.) and accurately locate the damaged area, providing accurate basic data for subsequent monitoring.

[0135] Function: This core step uses a deep learning model to analyze images, identify the type and location of stone carving defects, automatically mark the defective areas, and provide detailed defect information for subsequent data recording.

[0136] Relationship: The disease detection step generates the recognition results of disease categories and locations based on the preprocessed images. This result will be used for data storage and display and is the core output of the entire system.

[0137] Step 4. Data storage and update

[0138] Conditions: The server receives the results of disease detection and automatically stores information such as the type, location, and time of the disease in the database. The database is required to support multiple accesses and rapid updates to facilitate future queries and analysis.

[0139] Intelligent data monitoring and long-term analysis: By storing the data uploaded by tourists and the disease identification results of the server in the database, the management end can perform time series analysis on the disease data to monitor the development trend of the disease. Machine learning algorithms are introduced to analyze and predict the disease time series data, evaluate the risk of stone carving disease expansion, and form a scientific basis for protection measures.

[0140] Closed-loop data interaction between the visitor side and the management side: The visitor side, server, and management side together form a closed loop of data collection, identification, monitoring, and feedback, which enhances the real-time and interactive nature of disease identification data. The visitor side not only displays the stone carving background and disease information to tourists, but also supports the management side's monitoring and protection through the disease data it captures.

[0141] Function: The database records each stone carving image and its disease information as basic data for long-term disease monitoring of cultural relics, enabling cultural relics protection personnel to query and analyze historical data and obtain disease development trends.

[0142] Relationship: The data storage step is the output of disease detection results, forming data accumulation, which provides support for cultural heritage protection personnel to monitor and study disease development. Data updates provide continuous data input for the management end.

[0143] Step 5. Data Analysis and Prediction

[0144] Conditions: The system receives disease data from the database at the management end, and uses data analysis algorithms (such as time series analysis and regression models) to perform statistical and predictive analysis on disease types and changing trends to ensure that the analysis model has sufficient historical data support and accuracy.

[0145] Data storage and analysis: Use MySQL database to store stone carving images and identification information, and use Python data analysis library to perform statistical analysis and predictive modeling of disease data.

[0146] Function: Data analysis can intuitively show the trends and changes in the occurrence of diseases, enabling cultural heritage protection personnel to prevent the spread of diseases in advance and provide a basis for decision-making in protecting stone carvings.

[0147] Relationship: Data analysis and prediction rely on the historical records of data storage. Through statistics and trend prediction of data, the development dynamics of diseases can be displayed, enabling cultural heritage protection personnel to formulate targeted protection measures.

[0148] Step 6. Results display and application

[0149] Conditions: The data results displayed on the management side require a simple interface, intuitive statistical charts, and provide comparisons of disease development over multiple time periods to ensure that disease information is visualized and easy to interpret.

[0150] Function: The result display enables cultural relics protection personnel to directly view the changing trends and statistical distribution of various diseases. Cultural relics protection personnel can make scientific decisions based on the analysis results, thereby improving the accuracy and efficiency of cultural relics protection.

[0151] Relationship: The result display is the final output of data analysis, enabling cultural conservation personnel to take corresponding conservation actions based on the analysis data. The display results also provide feedback to tourists, helping them understand the cultural background of the stone carvings and the current conservation status.

[0152] The above steps are progressive, from image acquisition to data storage and analysis, and then to display and application, forming a complete closed loop from data input to protection decision-making. Each step is connected to each other under the premise that the conditions are met, providing support for the system to efficiently identify and monitor stone carving diseases.

[0153] System structure diagram Figure 1 As shown, the data flow relationship between the visitor terminal, the server, the target detection model and the management terminal is shown. The present invention includes the visitor terminal, the server, the target detection model and the management terminal, and the structural relationship is as follows:

[0154] Visitor side: Visitors take photos of stone carvings and upload them through the mini program. The system automatically identifies the stone carvings based on the photos and returns the historical background and damage status of the stone carvings.

[0155] Server: Receives photos of stone carvings uploaded by tourists and passes them to the target detection model. It also stores the recognition results and updates the database for further disease analysis and long-term monitoring.

[0156] Object detection model: Stone carvings and defects are detected based on the improved Faster R-CNN model to identify the type and location of defects.

[0157] Management side: Cultural relics protection personnel can view real-time identification data through the management side, including images of stone carvings and distribution statistics of disease types, as well as long-term data analysis.

[0158] Example 1: Visitor terminal identification process

[0159] Flowchart as Figure 2 The figure shows the process of tourists taking photos and uploading them to stone carving disease identification. The specific steps are as follows:

[0160] Photo upload: Visitors can take photos of the stone carvings with their mobile phones in the stone carving scenic area and upload them to the server through the system.

[0161] Image preprocessing: The server performs basic preprocessing on photos, including image cropping, noise removal, and clarity enhancement, to improve the recognition accuracy of the detection model.

[0162] Identification of stone carvings and defects: The processed photos are input into the target detection model. The model uses the feature extraction network to analyze the image, extract the features of the stone carvings and defects, and output the defect category (such as peeling, cracks, etc.) and location.

[0163] Result feedback: After the recognition is completed, the system will feedback the cultural background information of the stone carving (such as the age of the stone carving, the author, the content, etc.) and the damage information (type and degree of damage, etc.) to the tourist end.

[0164] Example 2: Management-side disease data monitoring process

[0165] The management end disease data monitoring flow chart is as follows Figure 3 As shown in the figure, the whole process of disease data from storage, statistics to analysis and prediction is demonstrated. The specific steps are as follows:

[0166] Data reception and storage: The server stores the photos uploaded by each visitor and the identified disease results into the database, forming a disease data sequence arranged in chronological order.

[0167] Statistical analysis of disease types: The management end conducts statistical analysis on the disease types in the database, generates disease change trend charts for different periods, and supports prediction of the incidence of different types of diseases.

[0168] Historical data analysis and prediction: The system uses machine learning algorithms to perform regression analysis and trend prediction on historical disease data, assess the risk of disease expansion, and provide reference for cultural heritage protection personnel.

[0169] Result display and application: The disease analysis results are presented in visual charts on the management side, and cultural relics protection personnel can adjust cultural relics protection measures based on the results.

[0170] System interface design: The interface on the visitor side is simple and intuitive, providing textual introductions to stone carving cultural information and image annotations of diseases; the management side provides chart statistics functions to support comparative analysis of disease changes in different time periods.

[0171] Through the above scheme, the present invention can effectively use tourists' photo data to carry out automatic disease monitoring, and realize the intelligent and convenient identification and trend analysis of cultural relics diseases.

[0172] The core invention of the present invention lies in the automatic identification and monitoring of cliff carving diseases based on the improved Faster R-CNN model. This technology breaks through the limitations of traditional manual identification and single detection, and can realize efficient, automatic and dynamic monitoring of cliff carving diseases.

[0173] This invention transforms the detection of cliff stone defects from a single manual operation to an automated and systematic one, bringing important technological innovation and application value to the protection of cultural heritage. Through technological innovation, this patent not only serves the intelligent protection of cultural heritage, but also enriches the experience of tourists. It conforms to the general trend of the integration of culture and tourism, and provides a demonstration effect for technology to empower cultural relics protection and cultural communication.

[0174] In the process of achieving the goal of the present invention - automatically identifying cliff carvings and their defects and dynamically monitoring the development trend of the defects - in addition to the image recognition technology based on the improved Faster R-CNN model adopted by the present invention, there are also the following alternatives that can achieve similar goals:

[0175] 1. Real-time recognition alternative based on YOLO model

[0176] YOLO (You Only Look Once) is a real-time model widely used in object detection. Compared with Faster R-CNN, YOLO has a higher speed advantage and can process uploaded photos in a near real-time manner. If the YOLO model is used instead of Faster R-CNN, the system can respond to tourists' recognition needs more quickly and find a balance between recognition accuracy and efficiency. This alternative is suitable for use in scenarios with high processing speed requirements, but it may have a certain impact on accuracy.

[0177] 2. Alternatives to Multimodal Fusion Methods

[0178] By combining image recognition with multimodal data such as text and 3D point cloud, the recognition effect of stone carvings and defects can be improved. For example, by combining 3D point cloud data from laser scanning and comparing data from multiple time points, subtle changes in stone carving defects can be detected more accurately. This method can be used as a supplement or alternative to the present invention, but it requires additional hardware support and more complex data processing.

[0179] 3. Alternatives based on semantic segmentation

[0180] Using semantic segmentation models (such as U-Net or DeepLab) can finely segment the diseased area, which is suitable for applications that require accurate diseased area positioning. Unlike methods based on target detection, semantic segmentation models can provide more detailed information on the distribution of diseased areas and are suitable for detailed analysis of disease extension areas. This alternative has advantages when fine-grained detection is required, but may not be as versatile and real-time as target detection models.

[0181] 4. Expert system combined with automated data annotation alternatives

[0182] Based on image recognition, the expert system is combined to automatically judge the defects of stone carvings. This solution can assist in the identification of defects through pre-entered rules and improve the accuracy of identification. If the amount of data collected is not enough to support the training of deep learning models, the reliability of the system can be enhanced through this alternative solution, but this method has low flexibility and adaptability.

[0183] The above alternatives each have their own advantages. The selection of specific solutions can be adjusted according to the actual situation of project requirements, but they can all achieve the purpose of the present invention to a certain extent.

[0184] The present invention automatically identifies stone carving photos based on a deep learning model, identifies the identity of a specific stone carving and its surface damage, on the one hand showing tourists detailed cultural background and historical value, enhancing their visiting experience; on the other hand, continuously accumulating damage data through stone carving photos taken by tourists, forming a dynamic monitoring data stream. The present invention combines the cliff stone carving damage identification results of the target detection model with tourists' interactive feedback, regularly analyzes and generates damage development trend reports, provides tourists and cultural relics protection personnel with convenient and efficient damage monitoring services, and realizes automated, intelligent, and visualized damage monitoring and historical data analysis.

[0185] What is described above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An automatic monitoring system for cliff carving defects, characterized in that: It includes the tourist end, server, target detection model and management end; on the tourist end, tourists take photos of stone carvings and upload them to the server, the server automatically identifies them based on the photos, and returns the historical background and disease status of the stone carvings; the server receives the photos of cliff carvings uploaded by tourists and passes them to the target detection model, and stores the recognition results and updates the database; the target detection model performs stone carving and disease detection based on the improved Faster R-CNN model, and identifies the type and location of the disease; cultural relics protection personnel view real-time recognition data through the management end, including the disease images of cliff carvings and the distribution statistics of disease types, as well as long-term data analysis.

2. A method for automatically monitoring cliff carving defects, using the automatic monitoring system for cliff carving defects as claimed in claim 1, characterized in that: The steps include: Step 1: The tourist takes a clear picture of the cliff carvings and uploads it to the server; Step 2: After receiving the uploaded image, the server pre-processes the image; Step 3: Optimize and improve the existing Faster R-CNN model to adapt it to the unique characteristics of the cliff carvings, which have complex textures and minute disease details; the target detection model includes the improved Faster R-CNN model; the improved Faster R-CNN model is trained to identify the types of diseases on the cliff carvings; the preprocessed image is input into the improved Faster R-CNN model for identification, and the diseased area is located by combining the region proposal network and the feature pyramid network, and the disease type, diseased area and disease time are formed into disease information; Step 4: The server receives and stores the disease information in the database; Step 5: Use data analysis algorithms to conduct statistical and predictive modeling of disease types and changing trends; Step 6: The server pushes and displays the changing trend and statistical distribution of cliff carving diseases to the management end.

3. The automatic monitoring method for cliff carving defects according to claim 2 is characterized in that: In step 3, the method of locating the diseased area by the region proposal network and the feature pyramid network includes the following steps: Step 1: Introduce the feature pyramid network and use multi-layer feature fusion to generate a multi-scale pyramid feature map. l The fusion process is shown in formula (1): P l =Conv(C l )+Upsample(P l+1 ) (1) Where: Conv(C l ) represents the convolution feature map C l Perform 1×1 convolution operation to unify the number of channels; the output feature map of the convolution layer can be represented by C2, C3, C4, and C5, where C2 is the shallow feature map with the highest resolution and C5 is the deep feature map with the strongest semantic information; Upsample(P l+1 ) represents the feature map P of the previous pyramid layer l+1 Upsample to the same resolution as C l Matching; Through the top-down feature fusion path, the high-semantic deep features are gradually combined with the high-resolution shallow features to generate the output feature maps P2, P3, P4, P5 of each layer of the feature pyramid; Each pyramid level feature P l The local feature map is further extracted through 3×3 convolution for subsequent target detection; Step 2: First, run the region proposal network on each layer of the feature pyramid network to generate multi-scale candidate region proposal boxes (Anchors); calculate the coverage target probability p and boundary regression offset t of each candidate region for each Anchor according to formula (2) x ,t y ,t w ,t h : {p,t x ,t y ,t w ,t h }=RPN(P l ) (2) In the formula, RPN(P l ) represents the feature map P at each layer of the feature pyramid network l Run the region proposal network to generate multi-scale candidate region proposal boxes; Anchor covers a variety of disease targets from small cracks to large-scale spalling; The anchor is then mapped back to the image coordinates, and the box with the highest confidence is selected from the overlapping candidate region proposal boxes according to the preset non-maximum suppression (NMS) algorithm; Step 3: Feature fusion and detection: Through the feature map provided by the feature pyramid network, regional features are extracted from the feature map that best suits the target scale of the disease. For small cracks, the high-resolution feature maps of P2 and P3 are used to enhance edge information. For color difference diseases, the deep semantic features and shallow detail features are combined to identify color differences. The final disease type classification and disease area boundary regression are optimized through formulas (3), (4), (5), and (6): Classification of losses by disease type L cls Evaluate the matching degree between the class probability predicted by the model and the true label, calculated by formula (3): where y i is the true category label, is the predicted category probability; The bounding box regression error is calculated by smoothing the L1 loss to avoid excessive punishment for large errors; the boundary regression loss L reg Calculated by formula (4): Where, t i is the boundary value of the real box (center point and size); is the boundary value of the prediction box. The loss function Calculated by formula (5): In the formula, x represents the difference (error) between the target value and the predicted value, which is calculated as follows: Where: t is the target value, such as the coordinate value of the actual position of the bounding box; is the predicted value, such as the bounding box position predicted by the model; |x|: represents the absolute value of the error; The total loss function L is calculated by formula (6): L=L cls +λL reg (6) Where λ is the weight factor for adjusting classification loss and regression loss; The total loss function L comprehensively weighs the classification accuracy and bounding box positioning accuracy.

4. The automatic monitoring method for cliff carving defects according to claim 2 is characterized in that: The preprocessing operation in step 2 includes one or more operations of image cropping, noise removal, contrast enhancement, image color correction, edge enhancement, gamma correction, distortion correction, and image resolution improvement.

5. The automatic monitoring method for cliff carving defects according to claim 2 is characterized in that: The types of damage in step 3 include one or more of peeling, cracks, weathering, fissures, dust accumulation, water rust crusting, local loss, surface dissolution, powdering, flaking, alkali and salt flooding, and lower plant damage.

6. The automatic monitoring method for cliff carving defects according to claim 2 is characterized in that: In step 5, the data analysis algorithm is time series analysis and regression model; the MySQL database is used to store cliff carvings images and disease information, and the Python data analysis library is used to perform statistical analysis and predictive modeling of disease information data.

7. The automatic monitoring method for cliff carving defects according to claim 2 is characterized in that: In step 3, the Faster R-CNN model used to identify the types of cliff carving damage is replaced by the YOLO model or the image recognition is fused with text and 3D point cloud multimodal data.

8. The automatic monitoring method for cliff carving defects according to claim 2 is characterized in that: In step 3, the Faster R-CNN model used to identify the types of cliff carving damage is replaced with a U-Net semantic segmentation model or a DeepLab semantic segmentation model.

9. The automatic monitoring method for cliff carving defects according to claim 2 is characterized in that: Step 3 also includes an expert system; the expert system is used to automatically judge the defects of the cliff carvings.

Images