A system and method for evaluating architectural heritage protection based on image recognition

Through the architectural heritage protection assessment system based on image recognition, the Faster R-CNN model is used to automatically identify building cracks, which solves the problems of inefficient and insufficient accuracy of building structure assessment in the existing technology, and achieves efficient and accurate building damage analysis and early warning.

CN117875549BActive Publication Date: 2024-11-05KUNMING UNIV OF SCI & TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202311863251.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-29
Publication Date
2024-11-05
Estimated Expiration
2043-12-29

AI Technical Summary

Technical Problem

The prior art is difficult to efficiently and accurately evaluate and monitor the integrity and damage conditions of building structures, especially the difficulty in detecting high places or difficult to access areas, resulting in inefficiency and costly.

Method used

Using an image recognition-based architectural heritage protection evaluation system, through data set processing, model construction and image recognition modules, Faster R-CNN deep learning model automatically recognizes building cracks and generates building evaluation index to achieve automatic identification of cracks on building surfaces and quantitative analysis of damage.

Benefits of technology

It greatly reduces errors caused by human factors, reduces the time and resources required for expert evaluation, improves the accuracy of evaluation, and promptly detects potential structural risks to prevent unnecessary damage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117875549B_ABST
    Figure CN117875549B_ABST
Patent Text Reader

Abstract

The present invention provides a system and method for building heritage protection and assessment based on image recognition, which relates to the technical field of building protection and assessment, and comprises a data set processing module, a data set marking module, a model building module, an image sampling module, an image recognition module and a building assessment module. The present invention uses a Faster R-CNN deep learning model, pre-processes and marks existing crack images, and constructs corresponding training data sets, so that the deep learning model system can automatically identify the existence of cracks and quantify the size of the cracks, realize automatic identification of cracks on the surface of buildings, and quantitatively analyze the degree of damage to the buildings through a building assessment index, which greatly reduces the errors caused by human factors and reduces the time and resources required for expert assessment. The building assessment index is a comprehensive analysis of the height, number, length and width of the cracks, rather than a single analysis, to ensure the accuracy of data analysis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of building protection assessment, and in particular to a building heritage protection assessment system and method based on image recognition. Background Art

[0002] At present, the main hazards to these buildings are cracks in the buildings, which may indicate potential structural problems. They not only affect the aesthetics of the buildings, but more importantly, they may indicate safety issues of the buildings. Over time, these cracks may expand, leading to water infiltration, and then causing more serious structural damage, such as corrosion of beams and columns, collapse of walls, etc. By scientifically analyzing and treating the cracks, the loss of cultural heritage can be minimized. Therefore, it is crucial to monitor and evaluate the cracks regularly.

[0003] At present, the main technical problem is how to efficiently and accurately evaluate and monitor the structural integrity and damage status of buildings. The current traditional method relies on on-site inspections by experts and manual recording of cracks and damage conditions. This is not only time-consuming and labor-intensive, but also difficult to achieve continuous monitoring. It is especially difficult to detect cracks at heights and locations that are difficult to reach.

[0004] Deficiencies of existing technologies: Currently, the assessment and damage monitoring of building structures mainly rely on on-site visual inspections and empirical judgments by experts, which is not only inefficient, but also limited in accuracy by the subjective experience of experts and the environmental conditions during the inspection. In addition, the geographical location of these buildings may be remote, making regular maintenance inspections expensive and difficult, especially for inspections in high or inaccessible areas. Therefore, it is of great significance to develop a system that can accurately and objectively conduct conservation assessments on architectural heritage.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a system and method for evaluating architectural heritage protection based on image recognition to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] An architectural heritage protection assessment system based on image recognition, comprising:

[0009] A data set processing module, the data set processing module is used to collect different types of crack images on the wall of the ancient building, and the collected crack images are pre-processed to form a training image data set of ancient building crack images;

[0010] A data set labeling module, which is used to label the images in the training image data set with positive and negative labels, where the positive label indicates an area with cracks and the negative label indicates an area without cracks, and to label the images with positive labels with data labels, where the data labels include the number of cracks and the total length and total width of the cracks in the image, and to store the labeled images in a database to form a training data set;

[0011] A model building module, wherein the model building module is used to build an object detection model based on the Faster R-CNN network, input the training data set into the object detection model, and compare the training results with the image annotations. When the loss function is less than a set accuracy threshold, the training of the object detection model is completed;

[0012] An image sampling module, wherein the image sampling module is used to collect surface images of ancient buildings, form a recognition data set with the collected image set to be recognized, and perform height marking on the collected image to be recognized. When marking the height, the actual height of the building scene photographed in the center of the image from the ground is calibrated as the height mark of the corresponding image;

[0013] An image recognition module, which uses the trained object detection model to identify the positive and negative labels and data labels in the image to be recognized, obtains the positive and negative labels and data label data of each image to be recognized, and classifies images with positive labels and images with negative labels;

[0014] A building assessment module is used to count the number of images containing positive labels and the number of images containing negative labels, as well as the number, length and width of cracks in each image containing positive labels, and process the analyzed data to generate a building assessment index, and transmit the building assessment index to the assessment warning module, through which the assessment warning module issues or does not issue a warning prompt for the building assessment index.

[0015] Furthermore, the images collected by the data set processing module are color images in RGB mode with the same size and resolution, and the specific logic of preprocessing the collected crack images includes:

[0016] The collected crack images are enlarged by randomly rotating them and doubling them. The angle range of random rotation is [-60°, 60°]. The enlarged images and the original collected images together constitute the preprocessing data set.

[0017] The RGB components of the pixels in the image of the preprocessed data set are weighted to obtain a grayscale image of the image, and the grayscale image is subjected to feature sharpening processing, that is, the grayscale value of each pixel in the grayscale image is set to the median of the grayscale values ​​of each adjacent pixel;

[0018] The image after feature sharpening is subjected to contrast enhancement processing, and the gain of the high-frequency part of the image is calculated to obtain an enhanced image. The enhanced image set is the training image data set.

[0019] Furthermore, when obtaining the grayscale image of the internal image of the preprocessed data set, the color value of each pixel in the RGB color channel is obtained, and the grayscale value of each pixel is generated according to the following calculation formula to form a grayscale image. The formula is:

[0020] g(i,j)=0.3R(i,j)+0.6G(i,j)+0.1B(i,j)

[0021] Where g(i, j) represents the grayscale value of the pixel in the i-th row and j-th column of the image, R(i, j), G(i, j) and B(i, j) represent the R, G and B color values ​​of the pixel in the i-th row and j-th column of the image in the RGB color channel respectively;

[0022] When performing feature sharpening processing on grayscale images, the median filter method is used to sharpen the image;

[0023] The specific logic for contrast enhancement of images is as follows:

[0024] The pixel point in the i-th row and j-th column of the image is marked as (i, j), and the (2n+1)×

[0025] The mean K and variance σ of the image within the (2n+1) range, the formula for calculating the mean is:

[0026]

[0027] Where K(i, j) represents the mean value of the image within the range of (2n+1)×(2n+1) centered at (i, j), and (p, q) represents the grayscale value of the pixel point of the image within the range of (2n+1)×(2n+1) centered at (p, q).

[0028] The variance calculation is based on the formula:

[0029]

[0030] Among them, (p, q)-K(i, j) represents the high-frequency part of the image, and the formula for calculating the gain of the high-frequency part of the image is:

[0031] F(i,j)=K(i,j)+α[,(p,q)-K(i,j)]

[0032] Wherein, F(i, j) represents the gray value of the pixel (i, j) after gain, α is the gain weight, and α>1.

[0033] Furthermore, the surface images of the ancient buildings to be evaluated collected by the image sampling module have the same size and resolution as the size and resolution of the images collected by the data set processing module, and when collecting, the surface images of the ancient buildings are collected layer by layer along the height of the ancient buildings, that is, when collecting each layer, a circle of images of ancient buildings at the same height are collected, and then the pictures of ancient buildings on the next layer are collected, and the interval height between adjacent collection layers is the height dimension of the surface images of the ancient buildings to be evaluated collected by the image sampling module.

[0034] Furthermore, in the object detection model constructed by the model construction module, the loss function used to determine whether the training of the object detection model is completed includes a classification loss function, a number loss function and a size loss function, and the classification loss function is:

[0035]

[0036] Among them, L cls represents the classification loss function, y represents the true label annotation, Represents the predicted probability, the values ​​of positive and negative labels are both greater than 0;

[0037] The number loss function is:

[0038]

[0039] Among them, L box Represents the number loss function, smooth L1 (x) represents the smooth L1 loss function, b represents the number of cracks in the data label in the image with positive label, They represent the predicted number of cracks respectively;

[0040] The size loss function is:

[0041] L size =(L actual -L predicted ) 2 +(W actual -W predicted ) 2

[0042] Among them, L size represents the size loss function, L actual , Lpredicted They represent the total length and total width of cracks in the data labels in the image with positive labels, respectively. actual , W predicted They represent the total length and total width of the predicted cracks, respectively;

[0043] When the classification loss function, number loss function, and size loss function are all less than the corresponding set accuracy thresholds, it means that the training of the object detection model is completed.

[0044] Furthermore, when generating the building evaluation index, the number of images with positive labels and the number of images with negative labels, the number, length and width of cracks inside the images with positive labels, and the height mark of the images with positive labels are comprehensively generated. The specific logic for generating the building evaluation index is:

[0045]

[0046] Wherein, M and N represent the number of images with positive labels and the number of images with negative labels, respectively; Tf(m) represents the risk index of the mth image with positive labels, 1≤m≤M, m is a positive integer; JZf represents the architectural assessment index of the ancient building to be assessed; β and γ represent the height weight coefficient and the risk weight coefficient, respectively, and β>1, γ>0; Lc(m) and Lk(m) represent the total length and total width of the cracks in the mth image with positive labels, respectively; Ls(m) represents the number of cracks in the mth image with positive labels; and Gd(m) represents the height of the mth image with positive labels when it is collected.

[0047] Furthermore, the assessment and early warning module compares and analyzes the building assessment index with a preset building assessment reference threshold. If the building assessment index is greater than or equal to the building assessment reference threshold, a high-risk level signal is generated, and the assessment and early warning module issues an early warning prompt. If the building assessment index is less than the building assessment reference threshold, a safety level signal is generated, and the assessment and early warning module does not issue an early warning prompt.

[0048] The present invention further provides a method for assessing the protection of architectural heritage based on image recognition, the method being performed by the above-mentioned protection assessment system, comprising:

[0049] Collect different types of crack images on the walls of ancient buildings, and pre-process the collected crack images to form a training image dataset of ancient building crack images;

[0050] The images in the training image data set are labeled with positive and negative labels, where the positive label indicates an area with cracks and the negative label indicates an area without cracks, and the images with positive labels are labeled with data labels, where the data labels include the number of cracks and the length and width of each crack, and the labeled images are stored in a database to form a training data set;

[0051] Build an object detection model based on the Faster R-CNN network, input the training data set into the object detection model, and compare the training results with the image annotations. When the loss function is less than the set accuracy threshold, the training of the object detection model is completed.

[0052] The surface images of the ancient buildings are collected, and the collected images to be identified are used to form an identification data set. The collected images to be identified are height-marked. When the height is marked, the actual height of the building scene photographed in the center of the image from the ground is calibrated as the height mark of the corresponding image;

[0053] Use the trained object detection model to identify the positive and negative label annotations and data labels in the image to be identified, obtain the positive and negative label annotations and data label data of each image to be identified, and classify the images with positive labels and the images with negative labels;

[0054] The number of images with positive labels and images with negative labels, as well as the number, length and width of cracks in each image with positive labels are counted, and the analyzed data are processed to generate a building assessment index, and an early warning prompt is issued or not issued according to the building assessment index.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The present invention uses the Faster R-CNN deep learning model to preprocess and mark existing crack images and construct corresponding training data sets, so that the deep learning model system can automatically identify the existence of cracks and quantify the size of cracks, realize automatic identification of cracks on the surface of buildings, and quantitatively analyze the degree of damage to buildings through the building assessment index, which greatly reduces the errors caused by human factors and reduces the time and resources required for expert evaluation. The building assessment index is a comprehensive analysis of the height, number, length and width of the cracks, rather than a single analysis, which can effectively prevent accidental situations caused by a single analysis, ensure the accuracy of data analysis, and thus improve the accuracy of protection assessment;

[0057] The present invention can timely discover potential structural risks and issue warnings through the use of the building assessment index generated by the building assessment module and the assessment warning module, thereby protecting the architectural heritage from unnecessary damage. The design of the assessment warning module can timely indicate potential structural risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the overall system structure of the present invention;

[0059] Figure 2 Schematic diagram of the original image of the building crack in the present invention;

[0060] Figure 3 This is a schematic diagram of an image with positive and negative labels in the present invention;

[0061] Figure 4 This is a schematic diagram of an amplified image that is randomly rotated in the present invention;

[0062] Figure 5 This is a schematic diagram of an image after preprocessing in the present invention;

[0063] Figure 6 A schematic diagram of an image after being recognized by the image recognition module in the present invention;

[0064] Figure 7 Schematic diagram of the process of the evaluation method of the present invention. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0066] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0067] Example:

[0068] See also Figure 1-6 , the present invention provides a technical solution:

[0069] An architectural heritage protection and assessment system based on image recognition includes a data set processing module, a data set labeling module, a model building module, an image sampling module, an image recognition module and an architectural assessment module, wherein:

[0070] The data set processing module is used to collect different types of crack images on the walls of ancient buildings, and the collected crack images are preprocessed to form a training image data set of ancient building crack images.

[0071] In this embodiment, the sources of the crack images of the walls of the ancient buildings mainly include field shots and image data that have been publicly disclosed on the Internet. Different types of crack images refer to cracks in ancient buildings of different shapes, structures, lengths, and widths. The training image data set contains many different types of crack images, including images of ancient buildings without cracks.

[0072] Furthermore, the images collected by the data set processing module are color images in RGB mode with the same size and resolution. The uniform image size and resolution ensure the consistency of data input, which can reduce the preprocessing steps in subsequent model training and ensure that all images are treated equally during feature extraction. It reduces the noise caused by image size changes, and thus can help improve the performance of the algorithm. The specific logic of preprocessing the collected crack images includes:

[0073] The collected crack images are amplified and randomly rotated to double the size. The angle range of the random rotation is [-60°, 60°]. The amplified images and the original collected images together constitute a preprocessing data set, which can meet the requirement of a large number of data samples for model training.

[0074] The RGB components of the pixels in the image of the preprocessed data set are weighted to obtain a grayscale image of the image, and the grayscale image is subjected to feature sharpening, that is, the grayscale value of each pixel in the grayscale image is set to the median of the grayscale values ​​of each adjacent pixel.

[0075] In this embodiment, when obtaining the grayscale image of the internal image of the preprocessed data set, the color value of each pixel in the RGB color channel is obtained, and the grayscale value of each pixel is generated according to the following calculation formula to form a grayscale image. Its essence is image graying, which converts the color image in RGB mode into a grayscale image. The purpose is to reduce the amount of calculation and time during deep learning of the model. The formula is:

[0076] g(i,j)=0.3R(i,j)+0.6G(i,j)+0.1B(i,j)

[0077] Among them, g(i, j) represents the grayscale value of the pixel in the i-th row and j-th column of the image, R(i, j), G(i, j) and B(i, j) represent the R, G and B color values ​​of the pixel in the i-th row and j-th column of the image in the RGB color channel respectively.

[0078] When performing feature sharpening processing on a grayscale image, the median filtering method is used to sharpen the image. The basic principle is to replace the grayscale value of a pixel in the image with the median of the grayscale values ​​around the pixel, thereby reducing the noise of the image and making the edge of the crack smoother, which is convenient for subsequent model learning and training.

[0079] The image after feature sharpening is subjected to contrast enhancement processing, and gain calculation is performed on the high-frequency part of the image to obtain an enhanced image. The enhanced image set is the training image data set. Contrast enhancement processing can help improve the quality of image data, making the features in the image more prominent and facilitating model learning. In this embodiment, gain processing is performed on the high-frequency part of the image, which actually emphasizes the details and edge information in the image, because these usually correspond to the high-frequency components of the image.

[0080] In this embodiment, the specific logic for performing contrast enhancement processing on the image is:

[0081] The pixel point in the i-th row and j-th column of the image is marked as (i, j), and the (2n+1)×

[0082] The mean K and variance σ of the image within the (2n+1) range, the formula for calculating the mean is:

[0083]

[0084] Where K(i, j) represents the mean value of the image within the range of (2n+1)×(2n+1) centered at (i, j), and (p, q) represents the grayscale value of the pixel point of the image within the range of (2n+1)×(2n+1) centered at (p, q).

[0085] The variance calculation is based on the formula:

[0086]

[0087] Among them, (p, q)-K(i, j) represents the high-frequency part of the image, which is equivalent to a filter of a pixel point. The formula for calculating the gain of the high-frequency part of the image is:

[0088] F(i,j)=K(i,j)+α[,(p,q)-K(i,j)]

[0089] Wherein, F(i, j) represents the gray value of the pixel (i, j) after gain, α is the gain weight, and α>1.

[0090] The data set labeling module is used to annotate the images within the training image data set with positive and negative labels, where the positive label represents an area with cracks and the negative label represents an area without cracks, and to annotate the images with positive labels with data labels, where the data labels include the number of cracks and the total length and total width of the cracks in the image. The annotated images are stored in a database to form a training data set.

[0091] In this embodiment, positive and negative labels are annotated using the labeling tool "Label img" to manually annotate all images with positive and negative labels. The positive label represents an area with cracks, and the negative label represents an area without cracks. Each image in the training data set corresponds to a label annotation and a data label. During training, the label annotation and data label are equivalent to the training result labels.

[0092] The model building module is used to build an object detection model based on the Faster R-CNN network, input the training data set into the object detection model, and compare the training results with the image annotations. When the loss function is less than the set accuracy threshold, the training of the object detection model is completed.

[0093] The object detection model based on the Faster R-CNN network is an advanced deep learning model that can accurately detect multiple objects in an image and classify each object. The main structure of Faster R-CNN includes a feature extraction network, a region proposal network, a RoI pooling network, a classification and bounding box regression network, and non-maximum suppression. When the object detection model based on the Faster R-CNN network is trained to identify cracks in an image, the cracks are regarded as a special type of object. The basic principle of the model to identify cracks is similar to other object detection tasks in the prior art.

[0094] Furthermore, in the training process of the object detection model based on the Faster R-CNN network, the goal is to minimize the loss function. The loss function measures the difference between the model prediction result and the true annotation, that is, the label annotation and the data label. For this embodiment, the object detection model loss function generally includes three parts: classification loss function, number loss function and size loss function.

[0095] In the object detection model constructed by the model construction module, the loss function used to determine whether the training of the object detection model is completed includes a classification loss function, a number loss function and a size loss function. The classification loss function is:

[0096]

[0097] Among them, L cls represents the classification loss function, y represents the true label annotation, Represents the predicted probability, the values ​​of positive and negative labels are both greater than 0;

[0098] The number loss function is:

[0099]

[0100] Among them, L box Represents the number loss function, smooth L1 (x) represents the smooth L1 loss function, b represents the number of cracks in the data label in the image with positive label, They represent the predicted number of cracks respectively;

[0101] The size loss function is:

[0102] L size =(L actual -L predicted ) 2 +(W actual -W predicted ) 2

[0103] Among them, L size represents the size loss function, L actual , L predicted They represent the total length and total width of cracks in the data labels in the image with positive labels, respectively. actual , W predicted They represent the total length and total width of the predicted cracks, respectively;

[0104] When the classification loss function, number loss function, and size loss function are all less than the corresponding set accuracy thresholds, the training of the object detection model is completed.

[0105] During the training process, the model weights are continuously updated through back propagation and gradient descent to minimize the loss function. As the training progresses, if the model learns better, the value of the loss function should gradually decrease. Therefore, if the classification loss function, number loss function, and size loss function are all less than the corresponding set accuracy thresholds, it reflects the improvement of the model in classification accuracy and positioning accuracy.

[0106] The image sampling module is used to collect surface images of ancient buildings, form a recognition data set with the collected image set to be recognized, and perform height marking on the collected images to be recognized. When height marking, the actual height of the building scene captured in the center of the image from the ground is calibrated as the height mark of the corresponding image.

[0107] Furthermore, the surface image of the ancient building to be evaluated collected by the image sampling module has the same size and resolution as the image collected by the data set processing module, and when collecting, the surface image of the ancient building is collected layer by layer along the height of the ancient building, that is, when collecting each layer, a circle of images of ancient buildings at the same height are collected, and then the pictures of ancient buildings on the next layer are collected. The interval height between adjacent collection layers is the height size of the surface image of the ancient building to be evaluated collected by the image sampling module, which can completely collect the surrounding images of the entire building to ensure the accuracy and objectivity of the subsequent evaluation.

[0108] The image recognition module uses the trained object detection model to identify the positive and negative label annotations and data labels inside the image to be identified, obtains the positive and negative label annotations and data label data of each image to be identified, and classifies images containing positive labels and images containing negative labels.

[0109] In this embodiment, based on the positive and negative label annotations, the collected images can be divided into two categories, one is the positive label, representing the area with cracks, and the other is the negative label, representing the area without cracks. The data label is only available for the images annotated with the positive label. The data of the data label includes the number of cracks, the total length and total width of the internal cracks. Since the height of each image has been marked when it was collected, the image annotated with the positive label after identification will have the number of cracks, the total length and total width of the internal cracks, and the height of the image.

[0110] Optionally, when using the trained object detection model to identify the collected image to be identified, preprocessing may be performed according to the description of the above image preprocessing in this embodiment.

[0111] The building assessment module is used to count the number of images containing positive labels and the number of images containing negative labels, as well as the number, length and width of cracks in each image containing positive labels, and process the analyzed data to generate a building assessment index, and transmit the building assessment index to the assessment warning module, through which a warning prompt is issued or not issued for the building assessment index.

[0112] In this embodiment, the Faster R-CNN deep learning model is used to preprocess and label existing crack images and construct corresponding training data sets, so that the deep learning model system can automatically identify the existence of cracks and quantify the size of cracks, thereby realizing automatic identification of cracks on the surface of buildings and quantitatively analyzing the degree of damage to buildings through the building assessment index. This greatly reduces the errors caused by human factors and reduces the time and resources required for expert evaluation.

[0113] Furthermore, when generating the building evaluation index, the number of images with positive labels and the number of images with negative labels, the number, length and width of cracks inside the images with positive labels, and the height mark of the images with positive labels are comprehensively generated. The specific logic for generating the building evaluation index is:

[0114]

[0115] Wherein, M and N represent the number of images with positive labels and the number of images with negative labels, respectively; Tf(m) represents the risk index of the mth image with positive labels, 1≤m≤M, m is a positive integer; JZf represents the architectural assessment index of the ancient building to be assessed; β and γ represent the height weight coefficient and the risk weight coefficient, respectively, and β>1, γ>0; Lc(m) and Lk(m) represent the total length and total width of the cracks in the mth image with positive labels, respectively; Ls(m) represents the number of cracks in the mth image with positive labels; and Gd(m) represents the height of the mth image with positive labels when it is collected.

[0116] For the risk index of the mth image with a positive label, when the number of cracks in the image with a positive label is greater, Tf(m) is greater, indicating that the structure of the building has been damaged. When the total length and total width of the cracks in the image with a positive label are greater, Tf(m) is greater, indicating that the gap is larger. At this time, the increase in the gap indicates that the degree of damage to the building is higher. When the height at which the image is collected is lower, Tf(m) is greater, indicating that the gap is located at the bottom of the building. Since the bottom of the building bears a greater supporting force, the smaller Gd(m), the higher the possibility of danger to the building.

[0117] For the building assessment index, since the sum of the number of images containing positive labels and the number of images containing negative labels is fixed, at this time, if the number of images containing positive labels is more, the sum of the risk indexes of the corresponding images is more, and the building assessment index is larger, which indicates that the result of the building assessment is worse. The building assessment index of this embodiment is not only related to the crack parameters in each collected image, but also to the proportion of the number of images with cracks. The building assessment index is a comprehensive analysis of the height, number, length and width of the cracks, rather than a single analysis, which can effectively prevent accidental situations that occur in a single analysis, ensure the accuracy of data analysis, and thus improve the accuracy of protection assessment.

[0118] Furthermore, the assessment and early warning module compares and analyzes the building assessment index with a preset building assessment reference threshold. If the building assessment index is greater than or equal to the building assessment reference threshold, a high-risk level signal is generated, and the assessment and early warning module issues an early warning prompt. If the building assessment index is less than the building assessment reference threshold, a safety level signal is generated, and the assessment and early warning module does not issue an early warning prompt.

[0119] In this embodiment, the building assessment index generated by the building assessment module and the use of the assessment warning module can timely discover potential structural risks and issue warnings, thereby protecting the architectural heritage from unnecessary damage. The design of the assessment warning module can timely indicate potential structural risks.

[0120] See also Figure 7 The present invention further provides a method for assessing the protection of architectural heritage based on image recognition, the method being executed by the above-mentioned protection assessment system, comprising:

[0121] Step 1: Collect different types of crack images on the walls of ancient buildings, and pre-process the collected crack images to form a training image dataset of ancient building crack images;

[0122] Step 2: Annotate the images in the training image dataset with positive and negative labels, where the positive label indicates the area with cracks and the negative label indicates the area without cracks. Annotate the images with positive labels with data labels, including the number of cracks and the length and width of each crack. Store the annotated images in the database to form the training dataset.

[0123] Step 3: Build an object detection model based on the Faster R-CNN network, input the training data set into the object detection model, and compare the training results with the image annotations. When the loss function is less than the set accuracy threshold, the training of the object detection model is completed;

[0124] Step 4: Collect surface images of ancient buildings, form a recognition data set with the collected image set to be recognized, and mark the height of the collected image to be recognized. When marking the height, the actual height of the building scene photographed in the center of the image from the ground is calibrated as the height mark of the corresponding image;

[0125] Step 5: Use the trained object detection model to identify the positive and negative label annotations and data labels in the image to be identified, obtain the positive and negative label annotations and data label data of each image to be identified, and classify the images with positive labels and the images with negative labels;

[0126] Step 6: Count the number of images with positive labels and images with negative labels, as well as the number, length and width of cracks in each image with positive labels, process the analyzed data, generate a building assessment index, and issue or not issue a warning prompt based on the building assessment index.

[0127] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0128] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by computer software, electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A building heritage protection assessment system based on image recognition, characterized in that: include: A data set processing module, the data set processing module is used to collect different types of crack images on the wall of the ancient building, and the collected crack images are pre-processed to form a training image data set of ancient building crack images; A data set labeling module, which is used to label the images in the training image data set with positive and negative labels, where the positive label indicates an area with cracks and the negative label indicates an area without cracks, and to label the images with positive labels with data labels, where the data labels include the number of cracks and the total length and total width of the cracks in the image, and to store the labeled images in a database to form a training data set; A model building module, wherein the model building module is used to build an object detection model based on the Faster R-CNN network, input the training data set into the object detection model, and compare the training results with the image annotations. When the loss function is less than a set accuracy threshold, the training of the object detection model is completed; An image sampling module, wherein the image sampling module is used to collect surface images of ancient buildings, form a recognition data set with the collected image set to be recognized, and perform height marking on the collected image to be recognized. When marking the height, the actual height of the building scene photographed in the center of the image from the ground is calibrated as the height mark of the corresponding image; An image recognition module, which uses the trained object detection model to identify the positive and negative labels and data labels in the image to be recognized, obtains the positive and negative labels and data label data of each image to be recognized, and classifies images with positive labels and images with negative labels; A building assessment module, wherein the building assessment module is used to count the number of images containing positive labels and the number of images containing negative labels, as well as the number, length and width of cracks in each image containing positive labels, and to process the analyzed data to generate a building assessment index, and to transmit the building assessment index to the assessment warning module, and to issue or not issue a warning prompt for the building assessment index through the assessment warning module; When generating the building evaluation index, the number of images with positive labels and the number of images with negative labels, the number of cracks inside the images with positive labels, the total length and total width, and the height mark of the images with positive labels are comprehensively generated. The specific logic for generating the building evaluation index is as follows: Wherein, M and N represent the number of images with positive labels and the number of images with negative labels, respectively; Tf(m) represents the risk index of the mth image with positive labels, 1≤m≤M, m is a positive integer; JZf represents the architectural assessment index of the ancient building to be assessed; β and γ represent the height weight coefficient and the risk weight coefficient, respectively, and β>1, γ>0; Lc(m) and Lk(m) represent the total length and total width of the cracks in the mth image with positive labels, respectively; Ls(m) represents the number of cracks in the mth image with positive labels; Gd(m) represents the height of the mth image with positive labels when it is collected; The assessment and early warning module compares and analyzes the building assessment index with a preset building assessment reference threshold. If the building assessment index is greater than or equal to the building assessment reference threshold, a high-risk level signal is generated, and the assessment and early warning module issues an early warning prompt. If the building assessment index is less than the building assessment reference threshold, a safety level signal is generated, and the assessment and early warning module does not issue an early warning prompt.

2. The architectural heritage protection and assessment system based on image recognition according to claim 1 is characterized by: The images collected by the data set processing module are color images in RGB mode with the same size and resolution, and the specific logic of preprocessing the collected crack images includes: The collected crack images are enlarged by randomly rotating them and doubling them. The angle range of random rotation is [-60°, 60°]. The enlarged images and the original collected images together constitute the preprocessing data set. The RGB components of the pixels in the image of the preprocessed data set are weighted to obtain a grayscale image of the image, and the grayscale image is subjected to feature sharpening processing, that is, the grayscale value of each pixel in the grayscale image is set to the median of the grayscale values ​​of each adjacent pixel; The image after feature sharpening is subjected to contrast enhancement processing, and the gain of the high-frequency part of the image is calculated to obtain an enhanced image. The enhanced image set is the training image data set.

3. The architectural heritage protection and assessment system based on image recognition according to claim 2 is characterized by: When obtaining the grayscale image of the internal image of the preprocessed data set, the color value of each pixel in the RGB color channel is obtained, and the grayscale value of each pixel is generated according to the following calculation formula to form a grayscale image. The formula is: g(i,j)=0.3R(i,j)+0.6G(i,j)+0.1B(i,j) Where g(i, j) represents the grayscale value of the pixel in the i-th row and j-th column of the image, R(i, j), G(i, j) and B(i, j) represent the R, G and B color values ​​of the pixel in the i-th row and j-th column of the image in the RGB color channel respectively; When performing feature sharpening processing on grayscale images, the median filter method is used to sharpen the image; The specific logic for contrast enhancement of images is as follows: The pixel point in the i-th row and j-th column of the image is marked as (i, j), and the mean K and variance σ of the image within the range (2n+1)×(2n+1) centered at the pixel point (i, j) are calculated. The formula for the mean calculation is: Where K(i, j) represents the mean value of the image within the range of (2n+1)×(2n+1) centered at (i, j), and (p, q) represents the grayscale value of the pixel point of the image within the range of (2n+1)×(2n+1) centered at (p, q). The variance calculation is based on the formula: Among them, (p, q)-K(i, j) represents the high-frequency part of the image, and the formula for calculating the gain of the high-frequency part of the image is: F(i,j)=K(i,j)+α[(p,q)-K(i,j)] Wherein, F(i, j) represents the gray value of the pixel (i, j) after gain, α is the gain weight, and α>1.

4. The architectural heritage protection and assessment system based on image recognition according to claim 2 is characterized by: The surface image of the ancient building to be evaluated collected by the image sampling module has the same size and resolution as the size and resolution of the image collected by the data set processing module. When collecting, the surface image of the ancient building is collected layer by layer along the height of the ancient building, that is, when collecting each layer, a circle of images of ancient buildings at the same height are collected, and then the pictures of ancient buildings in the next layer are collected. The interval height between adjacent collection layers is the height size of the surface image of the ancient building to be evaluated collected by the image sampling module.

5. The architectural heritage protection and assessment system based on image recognition according to claim 1 is characterized by: In the object detection model constructed by the model construction module, the loss function used to determine whether the training of the object detection model is completed includes a classification loss function, a number loss function and a size loss function. The classification loss function is: Among them, L cls represents the classification loss function, y represents the true label annotation, Represents the predicted probability, the values ​​of positive and negative labels are both greater than 0; The number loss function is: Among them, L box Represents the number loss function, smooth L1 (x) represents the smooth L1 loss function, b represents the number of cracks in the data label in the image with positive label, They represent the predicted number of cracks respectively; The size loss function is: L size =(L actual -L predicted ) 2 +(W actual -W predicted ) 2 Among them, L size represents the size loss function, L actual , L predicted They represent the total length and total width of cracks in the data labels in the image with positive labels, respectively. actual , W predicted They represent the total length and total width of the predicted cracks, respectively; When the classification loss function, number loss function, and size loss function are all less than the corresponding set accuracy thresholds, it means that the training of the object detection model is completed.

6. A method for assessing architectural heritage protection based on image recognition, characterized in that: The assessment method is performed by the protection assessment system according to any one of claims 1 to 5, comprising: Collect different types of crack images on the walls of ancient buildings, and pre-process the collected crack images to form a training image dataset of ancient building crack images; The images in the training image data set are labeled with positive and negative labels, where the positive label indicates an area with cracks and the negative label indicates an area without cracks, and the images with positive labels are labeled with data labels, where the data labels include the number of cracks and the length and width of each crack, and the labeled images are stored in a database to form a training data set; Build an object detection model based on the Faster R-CNN network, input the training data set into the object detection model, and compare the training results with the image annotations. When the loss function is less than the set accuracy threshold, the training of the object detection model is completed. The surface images of the ancient buildings are collected, and the collected images to be identified are used to form an identification data set. The collected images to be identified are height-marked. When the height is marked, the actual height of the building scene photographed in the center of the image from the ground is calibrated as the height mark of the corresponding image; Use the trained object detection model to identify the positive and negative label annotations and data labels in the image to be identified, obtain the positive and negative label annotations and data label data of each image to be identified, and classify the images with positive labels and the images with negative labels; The number of images with positive labels and images with negative labels, as well as the number, length and width of cracks in each image with positive labels are counted, and the analyzed data are processed to generate a building assessment index, and an early warning prompt is issued or not issued according to the building assessment index.

Citation Information

Cited By

  • Method for image analysis for diagnosing condition of buildings and structures

    RU2858387C1