Apparatus and method for determining deformation and displacement of cultural asset

Abstract

The architectural cultural property deformation determination device according to the present invention comprises a data input unit that receives an image set of buildings, a neural network selection unit that selects a plurality of convolutional neural networks to form an ensemble classification model based on the type of deformation of the buildings determined in the input image set of buildings, an ensemble modeling unit composed of the plurality of convolutional neural networks selected by the neural network selection unit, an ensemble processing unit that combines a plurality of feature vectors derived through the ensemble modeling unit, a determination unit that derives probability values ​​for a plurality of classes from the combined feature vectors, and a variation amount output unit that derives a variation amount of the buildings based on the derived probability values, wherein the ensemble modeling unit receives the image set of buildings and models it with the selected plurality of convolutional neural networks.

Description

Technology Field

[0001] The present invention relates to a device and method for determining deformation of a building, and more specifically, to a device and method for determining the occurrence and degree of deformation of an architectural cultural property using artificial intelligence. Background Technology

[0002] Generally, as time passes after construction, buildings may undergo deformation due to cracks caused by the aging and corrosion of building materials. In particular, architectural cultural heritage is highly susceptible to such deformation as wooden materials are frequently used. Furthermore, as the risk of earthquakes has increased recently, there is a very high possibility of deformation and displacement of architectural cultural heritage in the event of an earthquake (hereinafter, the deformation of the present invention is a concept that includes both deformation and displacement).

[0003] However, since the degree of such deformation occurring in architectural cultural heritage is minute unless it is caused by sudden natural disasters such as earthquakes or floods, it is necessary to observe and collect relevant data accumulated over a long period to detect whether deformation has occurred.

[0004] To identify deformation in such architectural cultural heritage, visual observation and / or the attachment of sensors to various parts of the heritage have been used to check for deformation; however, there are issues regarding the inaccuracy of observation results and increased maintenance costs for the sensors.

[0005] Meanwhile, as research on technology capable of detecting minute deformations in captured objects using artificial intelligence-based image recognition and analysis techniques is currently active, it is necessary to utilize this to determine whether architectural cultural heritage has undergone deformation. The problem to be solved

[0006] The objective of the present invention, which aims to solve the aforementioned problems, is to provide an apparatus and method for determining deformation of architectural cultural heritage by using artificial intelligence image analysis technology to detect whether deformation has occurred in architectural cultural heritage and to determine the degree of deformation.

[0007] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0008] A device for determining deformation of an architectural cultural property according to an embodiment of the present invention for achieving the above objective comprises: a data input unit for receiving a set of architectural images; a neural network selection unit for selecting a plurality of convolutional neural networks to form an ensemble classification model based on the type of deformation of the architectural image determined in the input set of architectural images; an ensemble modeling unit composed of the plurality of convolutional neural networks selected by the neural network selection unit; an ensemble processing unit for combining a plurality of feature vectors derived through the ensemble modeling unit; a judgment unit for deriving probability values ​​for a plurality of classes from the combined feature vectors; and a variation amount output unit for deriving a variation amount of the architectural image based on the derived probability values. The ensemble modeling unit receives the set of architectural images and models it using the selected plurality of convolutional neural networks.

[0009] A method for determining deformation of an architectural cultural property according to an embodiment of the present invention for achieving the above objective comprises the steps of: receiving a set of architectural images; constructing a convolutional ensemble classification model determined according to a part of the building identified in the set of architectural images; inputting the set of architectural images into the convolutional ensemble classification model to derive a plurality of feature vectors and then combining them; inputting the combined feature vectors into a softmax function to derive probability values ​​for a plurality of classes; and deriving a variation amount of the building based on the derived probability values. Effects of the invention

[0010] According to the present invention, by using artificial intelligence-based image analysis technology without using sensors that have cost and maintenance issues, it is possible to detect deformation of architectural cultural heritage at an early stage, as well as determine the degree and severity of the deformation, thereby allowing for the determination of priorities for damage restoration and increasing the efficiency of architectural cultural heritage management. Brief explanation of the drawing

[0011] FIG. 1 is a configuration diagram showing a device for determining deformation of architectural cultural property according to one embodiment of the present invention. FIG. 2 is a conceptual diagram showing a specific configuration example of a device for determining deformation of architectural cultural property according to one embodiment of the present invention. Figure 3 is a diagram showing the separation of wooden architectural cultural heritage. Figure 4 is a figure showing four classes of separation of wooden architectural cultural heritage. FIG. 5 is a conceptual diagram showing a specific configuration example of a device for determining deformation of architectural cultural property according to another embodiment of the present invention. Figure 6 is a diagram showing the separation of wooden architectural cultural heritage. Figure 7 is a figure showing the four classes of separation of wooden architectural cultural heritage. FIG. 8 is a conceptual diagram showing a specific configuration example of a device for determining deformation of architectural cultural property according to another embodiment of the present invention. Figure 9 is a diagram showing the separation of stone architectural cultural heritage. Figure 10 is a figure showing four classes of separation of stone architectural cultural heritage. FIG. 11 is a flowchart illustrating a method for determining deformation of architectural cultural property according to one embodiment of the present invention. Specific details for implementing the invention

[0012] The present invention is capable of various modifications and may have various embodiments, and some embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and the scope of the present invention should be understood to include all modifications, variations, equivalents, and substitutions that fall within the technical spirit of the present invention related to the embodiments.

[0013] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. The term "and / or" includes a combination of multiple related described items or any of the multiple related described items.

[0014] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. It should be understood that no other components exist in between only when it is stated that one component is "directly connected" or "directly connected" to another component.

[0015] The terms used in this application are used merely to describe the embodiments presented and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of other features, numbers, steps, actions, components, parts, or combinations thereof.

[0016] As used in this application, "the above" and similar designations may indicate both singular and plural forms. Furthermore, unless there is a description explicitly specifying the order of steps describing the method according to this disclosure, the described steps may be performed in the order intended to achieve the purpose. This disclosure is not limited by the order in which the described steps are described.

[0018] All terms used in this specification, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains, and should not be interpreted in an ideal or overly narrow formal sense; and where the meaning of any term is defined in this specification, that term shall be interpreted as defined.

[0019] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate an overall understanding of the present invention, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0021] The present invention is characterized by a configuration that, after augmenting a set of architectural images, derives probability values ​​for four classes using an ensemble model composed of different convolutional neural networks according to the image region of the architectural cultural property, assigns weights equal to or different to each probability value, and then adds the weighted probability values ​​for each class to derive a final amount of variation, thereby detecting deformation of the architectural cultural property at an early stage and determining the degree and severity of the deformation.

[0023] FIG. 1 is a configuration diagram of a device (100) for determining deformation of architectural cultural property according to one embodiment of the present invention.

[0024] A device for determining deformation of an architectural cultural property (100) according to one embodiment of the present invention comprises: a data input unit (110) for receiving a set of architectural images; a data preprocessing unit (150) for performing preprocessing on the set of architectural images; a neural network selection unit (120) for selecting a plurality of convolutional neural networks to form an ensemble classification model based on the type of deformation of the architectural image determined in the input set of architectural images; an ensemble modeling unit (130) composed of a plurality of convolutional neural networks selected by the neural network selection unit; an ensemble processing unit (160) for combining a plurality of feature vectors derived through the ensemble modeling unit; a judgment unit (170) for deriving probability values ​​for four classes from the combined feature vectors; and a deformation amount output unit (180) for deriving a deformation amount of the architectural image based on the derived probability values.

[0025] A data input unit (110) of a device (100) for determining deformation of architectural cultural property according to one embodiment of the present invention receives a target building image data set (hereinafter referred to as the building image set).

[0026] A data preprocessing unit (150) of a device (100) for determining deformation of architectural cultural property according to one embodiment of the present invention performs preprocessing on an input set of architectural images. In most cases, since the size of the target architectural image data set is not large, it is desirable to perform preprocessing such as a data augmentation technique. A data augmentation technique is a technique that increases the number of artificial data sets by modifying the size, direction, position, brightness, etc., of the original data set.

[0027] The data preprocessing unit (150) of the present invention performs preprocessing on an input set of building images using augmentation techniques such as HorizontalFlip (flips the image left and right with a 50% probability), GaussianBlur (applies a Gaussian blur effect to the image with a 50% probability), HueSaturationValue (changes the hue and saturation of the image with a 50% probability), ColorJitter (changes the brightness, contrast, and saturation of the image with a 50% probability), and Affine (tilts the image to a range between -15 degrees and 15 degrees with a 50% probability).

[0028] Meanwhile, in the field of image recognition, there is an ensemble technique used to improve image classification performance. An ensemble technique is a method of making decisions by combining multiple opinions obtained from various experts in the most rational and efficient way; in the field of convolutional neural networks, ensemble methods combine the prediction values ​​output by multiple convolutional neural network models in an appropriate manner to produce higher accuracy. Generally, an ensemble technique consists of an ensemble generation step in which multiple convolutional neural network models are selected and configured, and an ensemble integration step in which the respective prediction values ​​from the selected and configured convolutional neural network models are combined.

[0029] In other words, ensemble techniques can be used to improve classification accuracy by combining multiple features of different convolutional neural network models into a single predictive feature, thereby preventing the risk of classification being performed using a single feature extracted from a single convolutional neural network model with low performance.

[0030] Ensemble techniques can be divided into feature ensembles and classifier ensembles depending on the level of integration. Feature ensembles involve the integration of a set of feature vectors that come in as input values ​​to the classifier for the final output, whereas classifier ensembles involve the integration of a set of outputs from the classifier where the final output value is determined by a voting method.

[0031] As will be described later, the feature vector set derived through the ensemble technique contains richer information about the building image than the output set of each classifier, so integration at this level provides better classification results; therefore, in the embodiments of the present invention, the feature ensemble technique is applied as the preferred ensemble technique.

[0032] A neural network selection unit (120) of a device (100) for determining deformation of an architectural cultural property according to one embodiment of the present invention selects appropriate convolutional neural networks according to the deformation prediction area of ​​a target building and transmits them to an ensemble modeling unit to be described later.

[0033] The convolutional neural network model used in the ensemble modeling unit (130) of the architectural cultural property deformation judgment device (100) according to one embodiment of the present invention is as follows.

[0034] 1) ResNet

[0035] ResNet (Residual Network) mitigates the gradient vanishing problem using residual learning, and various variations exist depending on the number of layers in the residual network. In the embodiments of the present invention, ResNet-50 and ResNet-101 were used, but the invention is not limited thereto.

[0036] 2) DenseNet

[0037] DenseNet (Dense Convolutional Network) requires fewer parameters than conventional CNN architectures because it does not learn feature maps redundantly, and it uses very narrow layers to reduce the number of parameters. There are four variations of DenseNet based on the number of layers: DenseNet-121, DenseNet-169, DenseNet-201, and DenseNet-264. In the embodiments of the present invention, two well-known variations, DenseNet-121 and DenseNet-169, were used, but the invention is not limited thereto.

[0038] 3) VGG

[0039] VGG has up to 19 layers, the convolutional layers use 3x3 convolutional filters, and pooling layers are placed between each group of two or three convolutional layers. In the embodiments of the present invention, VGG-16 and VGG-19, which have the best performance among the variant models of VGG, are used.

[0040] 4) AlexNet

[0041] AlexNet is an 8-layer convolutional neural network consisting of 5 convolutional layers, 3 fully connected layers, and max pooling layers between them. To speed up the training process, the ReLu activation function is applied to the fully connected layers and convolutional layers.

[0042] 5) Inception V3

[0043] Inception V3 is a model that decomposes convolutions with large filter sizes (e.g., 5x5 convolutional filters) into smaller convolutions (e.g., two 3x3 convolutional filters) and spatially factorizes standard convolutions into asymmetric convolutions.

[0044] 6) ResNeXt

[0045] ResNeXt was developed using a split-transform-merge strategy similar to that of the Inception module, which has residual blocks; the split-transform-merge strategy of the Inception module used in each block is utilized to enhance the representation of dense and large layers. The ResNeXt used in the embodiments of the present invention utilizes ResNeXt-50 and ResNeXt-101, which are variations based on the number of layers, but is not limited thereto.

[0046] 7) ShuffleNet V2

[0047] ShuffleNet is a computationally efficient convolutional neural network model designed for mobile devices that uses two operations: Pointwise Group Convolution and Channel Shuffle. It reduces computational costs by utilizing depth-separable convolution operations using 3x3 kernels. ShuffleNet V2 is a version that introduces additional highly efficient mechanisms to reduce memory access costs.

[0048] 8) MobileNet V2

[0049] MobileNet is a lightweight convolutional neural network model that uses depthwise separable convolution to reduce the computation of FLOPs and the number of parameters. MobileNet V2 utilizes two optimized mechanisms: a linear bottleneck that eliminates non-linearity in narrow layers and a reverse residual structure where shortest connections exist between thin layers.

[0050] 9) MnasNet

[0051] MnasNet is an automated neural architecture search network that measures latency in real time by running the model on mobile devices instead of using FLOPs.

[0052] The ensemble modeling unit (130) of the architectural cultural property deformation judgment device (100) according to one embodiment of the present invention implements an ensemble model using pre-trained convolutional neural networks selected by the neural network selection unit (120). The neural network selection unit (120) of the present invention implements an ensemble model by selecting the top three pre-trained convolutional neural network models that produce high performance in each type of deformation detection (e.g., sagging of roof eaves, separation in wooden and stone structures, etc.) among various convolutional neural networks.

[0053] For example, to detect separation deformation of a building as determined by a set of building images as shown in FIG. 3, the neural network selection unit (120) of the present invention, as shown in FIG. 2, selects a total of three pre-trained convolutional neural networks: ResNet-101 (130-1), DenseNet-169 (130-2), and MobileNet V2 (130-3). At this time, the selected ResNet-101, DenseNet-169, and MobileNet V2 are determined as artificial intelligence algorithms that exhibit the best performance according to a two-stage classification system for distinguishing normal and abnormal, and may vary depending on experimental results.

[0054] As another example, to detect sagging deformation of a building as determined by a set of building images as shown in FIG. 6, the neural network selection unit (120) of the present invention, as shown in FIG. 5, selects a total of three pre-trained convolutional neural networks: ResNet-101 (130-1), DenseNet-169 (130-2), and Inception V3 (130-3). At this time, the selected ResNet-101, DenseNet-169, and Inception V3 are determined as artificial intelligence algorithms that exhibit the best performance according to a two-stage classification system for distinguishing normal and abnormal, and may vary depending on experimental results.

[0055] As another example, to detect separation deformation occurring in a stone structure as determined by a set of building images as shown in FIG. 9, the neural network selection unit (120) of the present invention, as shown in FIG. 8, selects a total of three pre-trained convolutional neural networks: DenseNet-169 (130-1), ResNeXt-101 (130-2), and MobileNet V2 (130-3). At this time, the selected DenseNet-169, ResNeXt-101, and MobileNet V2 are determined as artificial intelligence algorithms that exhibit the best performance according to a two-stage classification system for distinguishing normal and abnormal, and may vary depending on experimental results.

[0056] Meanwhile, the convolutional neural network selected in the neural network selection unit (120) of the present invention may be changed by considering the type of detection target variation of the aforementioned target building through additional learning of the convolutional neural network.

[0057] In the ensemble modeling unit (130) of the architectural cultural property deformation determination device (100) according to one embodiment of the present invention, a preprocessed set of architectural images is input and input into each convolutional neural network to derive a feature vector for each convolutional neural network.

[0058] At this time, each of the three different feature vectors extracted from the ensemble modeling unit (130) of the present invention passes through a fully-connected layer to generate a feature vector with half the size. At this time, depending on the implementation, a dropout technique may be applied to avoid overfitting of the model when generating the feature vector.

[0059] An ensemble processing unit (160) of an architectural cultural property deformation determination device (100) according to one embodiment of the present invention combines feature vectors from an ensemble modeling unit (130) into a single feature vector and connects them again to a fully connected layer.

[0060] A judgment unit (170) of an architectural cultural property deformation judgment device (100) according to one embodiment of the present invention derives probability values ​​for a plurality of classes using a classification function. In the embodiment of the present invention, the classification function is preferably a softmax function, but is not necessarily limited thereto and may also use other classification functions such as ReLU. The judgment unit (170) of the present invention derives probability values ​​for four classes.

[0061] A class according to one embodiment of the present invention consists of a normal class, a low class (abnormal low), a middle class (abnormal middle), and a high class (abnormal high), as shown in FIGS. 4, 7, and 10. However, it is not limited thereto and may consist of more or fewer classes depending on the implementation.

[0062] That is, as shown in Fig. 4, when the type of deformation to be detected is separation of wooden architectural cultural heritage, the classes for the deformation level of separation can be configured into normal, low, middle, and high classes (where separation means that the supporting force between adjacent members weakens and expands outward, and is a significant damage factor related to structural imbalance that mainly occurs in masonry cultural heritage).

[0063] The normal class means there is no separation, the low class means there is a small level of separation, the middle class means there is an intermediate level of separation, and the high class means there is a high level of separation.

[0064] In addition, as shown in Fig. 7, even when the type of deformation to be detected is deflection of a wooden architectural cultural property, the classes for the level of deformation of deflection can be configured into normal, low, middle, and high classes (where deflection refers to the distance a specific point of the structure moves in the vertical direction (direction of load application) before and after deformation). The normal class means no deflection, the low class means a case where a small level of deflection exists, the middle class means a case where an intermediate level of deflection exists, and the high class means a case where a high level of deflection exists.

[0065] In addition, as shown in FIG. 10, even when the type of deformation to be detected is separation of a stone architectural cultural property, the classes for the deformation level of separation can be configured into a normal class, a low class, a middle class, and a high class. The normal class means no separation, the low class means a case where a small level of separation exists, the middle class means a case where an intermediate level of separation exists, and the high class means a case where a high level of separation exists.

[0066] In addition, when the type of deformation to be detected is the tilt of a wooden architectural cultural property, classes for the level of tilt deformation can be configured into normal, low, middle, and high classes. The normal class means no tilt, the low class means a case where a small level of tilt exists, the middle class means a case where a medium level of tilt exists, and the high class means a case where a high level of tilt exists.

[0067] The variation amount output unit (180) of the architectural cultural property deformation judgment device (100) according to one embodiment of the present invention derives the variation amount (Anomaly score) in the manner of Equation 1 using the probability values ​​and weights for the four classes derived from the judgment unit (170).

[0068]

[0069] Here, P_normal is the probability value for the normal class (weighted 0), p_low is the probability value for the low class (weighted 0.333), p_middle is the probability value for the middle class (weighted 0.666), and p_high is the probability value for the high class (weighted 1.0). The weights assigned to each class can vary depending on the implementation method.

[0071] The variation amount of the present invention can be classified into a total of four categories, such as abnormal high, abnormal medium, and abnormal low, depending on the degree of normal and abnormal, and the number of such classifications can be implemented differently depending on the target architectural cultural property.

[0072] The neural network learning unit (140) of the architectural cultural property deformation judgment device (100) according to one embodiment of the present invention provides a learned model to the convolutional neural network models (130-1, 130-2, 130-3) of the ensemble modeling unit (130) and provides an additional learned learning model.

[0073] Meanwhile, the architectural cultural property deformation determination device (100) according to one embodiment of the present invention can achieve good performance without building a vast amount of training data set by using transfer learning.

[0074] Strategy 1, a strategy related to neural network learning of the present invention, is a method of retraining the entire model from scratch by using only the structure of a pre-trained model. This method usually requires a large dataset.

[0075] Strategy 2 of transfer learning involves fixing a portion of the Convolutional Base while retraining the remaining layers and classifier. By utilizing the property that lower-level layers extract general features and higher-level layers extract specific and unique features, one can determine to what extent the neural network's parameters should be retrained.

[0076] Strategy 3 of transfer learning involves keeping the Convolutional Base as is and utilizing it as a feature extraction mechanism, while retraining only the classifier. This method is considered when computing power is insufficient, the dataset is small, or the pre-trained model is very similar to the dataset it has already been trained on.

[0077] FIG. 7 is a flowchart of a method for determining deformation of an architectural cultural property according to an embodiment of the present invention, which determines whether an architectural cultural property is deformed and the degree of severity using the architectural cultural property deformation determination device (100) of FIG. 1.

[0078] When a set of building images is input (S1110), a convolutional ensemble classification model is constructed based on the building parts identified (S1120) in the set of building images (S1130). At this time, preprocessing such as data augmentation can be performed on the input set of building images. Subsequently, the set of building images is input into the convolutional ensemble classification model (S1140) to derive multiple (preferably three) feature vectors and then combine them (S1150). Next, the combined feature vectors are input into a softmax function to derive probability values ​​for multiple (preferably four) classes (S1160).

[0079] At this time, when deriving probability values ​​for the four classes, the combined feature vector is input into a fully connected layer, and the output from the fully connected layer is input into a softmax function to derive probability values ​​for the four classes.

[0080] Based on the derived probability values, the variation amount of the building is derived (S1170). When deriving the variation amount of the building, weights are assigned to the four probability values ​​derived as in Equation 1, and the variation amount is derived by adding the weighted probability values. When deriving the variation amount by adding the weighted probability values, a variation level that is adaptively determined according to the target architectural cultural property is derived.

[0082] The above embodiments may be implemented using various forms of computing means including one or more processors, memory, and storage means. Additionally, a network interface connected to a wired or wireless network may be included. The processor may be a central processing unit or a semiconductor device that executes processing instructions stored in memory and / or storage units. The memory and storage units may include volatile storage media or non-volatile storage media. For example, the memory may include ROM and RAM. Accordingly, embodiments of the present invention may be implemented as a method implemented by a computer or as a non-transient computer-readable medium having computer-executable instructions stored on said computer. In one embodiment of the present invention, when executed by a processor, the computer-readable instructions may perform a method according to at least one aspect of the present invention.

[0084] The configuration of the present invention has been described in detail above through preferred embodiments of the present invention. However, the aforementioned embodiments are merely examples and do not limit the scope of the rights of the present invention. A person skilled in the art will be able to make various modifications and changes within the scope of the technical concept of the present invention from the teachings and suggestions of this specification. For example, the ensemble processing unit (160) and the judgment unit (170) may be implemented by integrating them into a single module or by dividing them into two or more devices. In addition, the convolutional neural networks input to the ensemble model may be fewer or more than three, and the classes derived after passing through the fully connected layer may be fewer or more than four. Accordingly, the scope of protection of the present invention should be determined by the description in the following claims.

Claims

Claim 1 A data input unit that receives a set of building images for a target building; a neural network selection unit that selects a plurality of convolutional neural networks for constructing an ensemble classification model from among pre-trained convolutional neural networks based on the performance regarding the types of deformation of the building identified in the building image set; an ensemble modeling unit that constructs the ensemble classification model using the plurality of convolutional neural networks selected by the neural network selection unit and inputs the building image set into the plurality of convolutional neural networks to derive a plurality of feature vectors; an ensemble processing unit that combines the plurality of feature vectors; and a judgment unit that derives probability values ​​for a plurality of classes from the combined feature vectors. An architectural cultural property deformation determination device comprising a deformation amount output unit that derives a deformation amount of the building based on the derived probability value, wherein the type of deformation includes any one of tilt, separation, and sagging of the building or a combination thereof, and wherein the neural network selection unit includes ResNet-101, DenseNet-169, and Inception V3 in the plurality of convolutional neural networks constituting the ensemble classification model when sagging is included in the type of deformation, and includes MobileNet V2 in the plurality of convolutional neural networks constituting the ensemble classification model when separation is included in the type of deformation. Claim 2 An architectural cultural property deformation determination device according to claim 1, further comprising a data preprocessing unit that performs preprocessing on the set of architectural images. Claim 3 delete Claim 4 An architectural cultural property deformation judgment device according to claim 1, wherein the judgment unit derives probability values ​​for the plurality of classes using a softmax function. Claim 5 An architectural cultural property deformation determination device according to claim 4, wherein the above-mentioned class is classified according to the deformation level and corresponds to at least one of a normal class, a low class, a middle class, and a high class. Claim 6 An architectural cultural property deformation determination device according to claim 4, wherein the variation amount output unit derives the variation amount by assigning weights to each of the probability values ​​for a plurality of derived classes and then adding the weighted probability values. Claim 7 In claim 6, the variation amount output unit derives a variation level adaptively determined according to the building, in a device for determining deformation of architectural cultural property. Claim 8 An architectural cultural property deformation determination device according to claim 2, wherein the data preprocessing unit augments the set of architectural images using at least one of HorizontalFlip, GaussianBlur, HueSaturationValue, ColorJitter, and Affine. Claim 9 A method performed by an architectural cultural heritage deformation determination device, comprising: receiving an image set of a building for a target building; selecting a plurality of convolutional neural networks from among pre-trained convolutional neural networks to construct a convolutional ensemble classification model based on the performance regarding the type of deformation of the building determined in the building image set; constructing the convolutional ensemble classification model with the plurality of convolutional neural networks; inputting the building image set into the convolutional ensemble classification model to derive a plurality of feature vectors and combining them; and inputting the combined plurality of feature vectors into a softmax function to derive probability values ​​for a plurality of classes. A method for determining deformation of an architectural cultural property, comprising the step of deriving a variation amount of the building based on the derived probability value, wherein the type of deformation includes any one of tilt, separation, and sagging of the building or a combination thereof, and the step of constructing the convolutional ensemble classification model includes, when the type of deformation includes sagging, including ResNet-101, DenseNet-169, and Inception V3 in the plurality of convolutional neural networks constituting the ensemble classification model, and when the type of deformation includes separation, including MobileNet V2 in the plurality of convolutional neural networks constituting the ensemble classification model. Claim 10 A method for determining deformation of an architectural cultural property according to claim 9, wherein the step of deriving the amount of variation of the above-mentioned building comprises: a step of assigning a weight to each of the derived probability values; and a step of adding the weighted probability values ​​to derive the amount of variation. Claim 11 A method for determining deformation of architectural cultural property according to claim 9, wherein the step of deriving probability values ​​for the plurality of classes comprises: a step of inputting the combined feature vector into a fully connected layer; and a step of inputting the output of the fully connected layer into the softmax function to derive probability values ​​for the plurality of classes. Claim 12 A method for determining deformation of architectural cultural property according to claim 11, wherein the step of inputting the output of the fully connected layer into the softmax function to derive probability values ​​for the plurality of classes includes the step of deriving probability values ​​for four classes consisting of a normal class, a low class, a middle class, and a high class, distinguished according to the deformation level. Claim 13 A method for determining deformation of architectural cultural property according to claim 9, wherein the step of receiving a set of architectural images includes the step of performing preprocessing on the received set of architectural images. Claim 14 A method for determining deformation of architectural cultural heritage according to claim 13, wherein the step of performing preprocessing on the input set of architectural images includes the step of augmenting the set of architectural images using at least one of HorizontalFlip, GaussianBlur, HueSaturationValue, ColorJitter, and Affine. Claim 15 A method for determining deformation of an architectural cultural property, wherein, in claim 10, the step of deriving a variation amount by adding the above-mentioned weighted probability values ​​includes the step of deriving a variation level that is adaptively determined according to the target architectural cultural property. Claim 16 delete Claim 17 delete

Images