Ancient building image semantic completion method based on attention mechanism and weighted concept lattice

Abstract

The application relates to an ancient building image semantic completion method based on an attention mechanism and a weighted concept lattice, and belongs to the technical field of image processing. The technical problems of significant feature loss and lack of relevance between semantic labels and image content caused by complex image structure and uneven feature distribution in the process of ancient building image labeling are solved. The solution is as follows: firstly, an initial label of an ancient building image is obtained by using a convolutional neural network with an integrated attention mechanism; then, a near neighbor label set is obtained, and the weight values of the labels in the near neighbor label set are calculated by using information entropy, so that a weighted concept lattice is constructed; finally, the weighted concept lattice is used for ancient building image semantic relevance analysis; according to an ancient building label connotation importance threshold value and a concept similarity, a final label set is obtained, and ancient building semantic labeling is realized. The application can objectively describe the importance of ancient building semantic labels in ancient building images, the accuracy of labeling is improved, and the effectiveness of the method is proved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a kind of ancient building image semantic complete method based on attention mechanism and weighted concept lattice. BACKGROUND

[0002] As a carrier of history, Chinese ancient architecture provides a lot of information for us to study the culture, architecture and technology of ancient China. With the deepening of social informatization, people have created and stored a large number of Chinese ancient architecture images. However, when using text to retrieve ancient architecture images, due to the existence of label missing and semantic noise in ancient architecture images, it also brings great challenges to the method of image retrieval relying on text. Therefore, it is very important to improve the efficiency of ancient architecture image retrieval and study an effective ancient architecture image semantic complete method.

[0003] The difficulty of ancient architecture image annotation lies in the effective extraction of image features. Traditional feature extraction methods usually extract low-level features such as color, texture and shape of ancient architecture images. However, due to the existence of semantic gap, these low-level visual features are often difficult to express high-level semantics. The method based on deep learning uses convolutional neural network to extract ancient architecture image features, which can easily establish the mapping relationship between image salient features and high-level semantics. However, due to the complex structure of ancient architecture images and uneven feature distribution, it is easy to cause the loss of salient features and reduce the extraction accuracy.

[0004] The essence of semantic annotation is to establish a mapping relationship between image targets and semantic labels. The method of improving ancient architecture image semantic labels based on concept lattice analyzes the implicit relationship between ancient architecture components and labels through concept lattice analysis, calculates the relevance of concepts to mine the potential semantics of ancient architecture images, so as to achieve the goal of ancient architecture image semantic complete. However, different ancient architecture semantic words play different roles in reflecting the style characteristics of ancient architecture. Such method does not reflect the importance of representative labels in these candidate labels. That is, it lacks to identify the importance of each semantic label reflecting the style characteristics of ancient architecture images, and the contribution degree of each type of label to the image is the same by default. However, different ancient architecture semantic words play different roles in reflecting the style characteristics of ancient architecture.

[0005] Due to the difficulty of effective extraction of image features and the difficulty of reflecting the importance of semantic labels in semantic annotation model, the accuracy, high precision and complete annotation of ancient architecture images are in urgent need. SUMMARY

[0006] The main purpose of the present application is to overcome the deficiencies in the prior art, solve the technical problems of significant feature loss caused by complex image structure and uneven feature distribution in the image annotation process of ancient architecture, and provide an ancient architecture image semantic completion method based on attention mechanism and weighted concept lattice.

[0007] The visual attention mechanism, as a resource allocation method in deep learning, gives different weights to the extracted features, and the size of the weight reflects the significance of the feature, so it can retain significant features while suppressing some unimportant information. Applying the visual attention mechanism to extract features of ancient architecture images can effectively extract important features such as the outline of ancient architecture in the image, thereby improving the annotation accuracy of ancient architecture images.

[0008] The number of semantic keywords is used to describe the image content and the feature information of the image, which often represents the main information of the image. However, in the general image annotation process, the frequency of image semantics does not necessarily represent its importance to the image content. For example, the same image label may correspond to multiple image region features, and the roof feature of ancient architecture is significant and is the most important region in the ancient architecture image. When annotating ancient architecture images, some labels in the image may appear multiple times in the candidate label set, and some labels may appear only once. However, the final annotation result will also remove duplicate label information and only keep single semantics. If researchers spend too much effort to study frequently appearing label information, they may overlook some important labels that appear less frequently, thereby affecting the accuracy of ancient architecture image annotation. Therefore, when annotating ancient architecture images, the weight of ancient architecture semantic labels is calculated by information entropy to determine the importance of each semantic label to the content of the ancient architecture image.

[0009] Weighted concept lattice is an extension structure of concept lattice, which assigns weights to the attribute connotation of concept nodes based on general concept lattice. Through the weight, the importance of attribute connotation in the whole knowledge can be described. This structure can effectively remove redundant information and extract important knowledge, and has effectiveness and practicality in knowledge representation.

[0010] The technical scheme adopted by the present application is:

[0011] The ancient architecture image semantic completion method based on attention mechanism and weighted concept lattice comprises the following steps:

[0012] S1, obtaining initial labels of ancient architecture images by using a convolutional neural network fused with attention mechanism;

[0013] S2, obtaining a near neighbor label set, and calculating the weight value of each label in the set by using information entropy, and then constructing a weighted concept lattice;

[0014] S3, ancient building image semantic correlation analysis using weighted concept lattice: according to the importance threshold of ancient building label connotation and concept similarity, the final label set is obtained.

[0015] Further, the step S1 comprises the following steps:

[0016] S1-1, extracting features of the ancient building image to be labeled by an attention operator network and a VGG16 network model convolution layer, obtaining a weight map and a feature map with unchanged size and channel number;

[0017] S1-2, fusing the weight map and the feature map obtained in step S1-1 to obtain an attention map capable of expressing the features of the ancient building image;

[0018] S1-3, obtaining the ancient building category label through the softmax layer in the CNN structure.

[0019] Further, the attention mechanism of the attention operator network is represented as:

[0020] f att =N(I); (1)

[0021]

[0022] In formula (1), the N(.) function is an attention operator network; I represents an input image, and the input image resolution is 224x224; f att represents the weight map matrix output after N(.) operation;

[0023] In formula (2), represents the attention saliency map obtained by multiplying the weight map and the feature map; F is the feature matrix of the input image I; represents the array element multiplication.

[0024] Further, the VGG16 network model has a total of 16 hidden layers, and the same size of convolution kernel and maximum pooling is used in the hidden layers. The hidden layers include 13 convolution layers and 3 fully connected layers, and the convolution layers include 1 input layer and 1 output layer. In order to increase the amount of ancient building image data, the ancient building image data is processed by rotation, scale change, mean value, etc., and the input ancient building image is processed by scale (224x224) and gray scale normalization.

[0025] Further, the size of the convolution kernel is 3x3, and the size of the maximum pooling is 2x2.

[0026] Further, the step S2 comprises the following steps:

[0027] S2-1, obtaining the initial neighbor set and the initial neighbor label set of the image I0 to be labeled;

[0028] The pre-trained VGG16 network model is used to extract 4096-dimensional feature vectors from the images. Euclidean distance is then used to calculate the visual similarity between the image to be labeled (I0) and the training images; a smaller distance indicates greater similarity. The formula for calculating the visual similarity between the image to be labeled (I0) and the training images is as follows:

[0029]

[0030] In equation (3), Sv represents visual similarity, and dist(I i I j Image I i and I j The Euclidean distance of the visual feature vectors; δ is a scalar, and the value of the scalar is set to the median of the image distance matrix;

[0031] The images are sorted in descending order of visual similarity, and the top n images with the highest similarity are selected as the nearest neighbor set and the nearest neighbor label set, where n is defined by the user. Finally, the nearest neighbor set and its nearest neighbor label set are merged with the initial category labels to form the final initial nearest neighbor set and initial nearest neighbor label set.

[0032] S2-2, Constructing a weighted concept lattice:

[0033] First, the weights of each attribute label in the initial nearest neighbor label set are calculated using information entropy:

[0034] Attribute set M = {t1, t2, ..., t} n}, for c w = (P, Q, w), and Q = t i (1≤i≤n), then weight(Q)=weight(t i )=H(t i ) = w i and w i Normalization is performed:

[0035]

[0036]

[0037] Among them, w i p(t) represents the weight of the single attribute connotation B. i / I j ) indicates that the object is I j The time includes the attribute t i The probability of for any t i H(t) i ) indicates I j Provided to attribute t iaverage information amount of the data set;

[0038] Secondly, according to the size of the attribute label weight, the semantic importance is analyzed: the arithmetic mean method is used to calculate the weight value of the multi-label connotation, which can reflect the overall importance of the multi-label connotation, and the weight value of the multi-label is obtained by calculating the weight value of the connotation attribute of each node in the lattice structure. w =(P,Q,w), and Q=t1∩t2∩…∩t n , t1, t2,..., t n ∈M, weight(t i )=w i (1≤i≤n), then the importance weight value of the multi-label connotation Q is defined as:

[0039]

[0040] Thirdly, by setting and adjusting the connotation importance threshold, the labels that do not meet the conditions are removed.

[0041] Finally, the weighted concept lattice is constructed by taking the set of neighboring images as the object and the set of neighboring labels with weight as the formal context input.

[0042] Further, the step S3 comprises the following steps:

[0043] Firstly, the similarity sim(a,b) between any two concept nodes a and b in the weighted concept lattice is:

[0044]

[0045] Wherein, the value of sim(a,b) is between 0 and 1, 1 represents complete mapping, and 0 represents wrong mapping; a∨b represents the intersection of a and b; | | represents the cardinality of the set; a LA is the lower approximation attribute set, that is, the largest definable set containing the concept a; a LA -b LA represents that the attribute set belongs to a LA , but does not belong to b LA ; α is the similarity measurement threshold, which is set to 0.5, when α≥0.5, there is a mapping relationship between the concept nodes a and b, and when α<0.5, there is no association between the concept nodes a and b;

[0046] Secondly, the visual similarity between the image to be labeled and the neighboring image is taken as the support degree of the neighboring image I m The support degree of each label in the candidate label set to the image to be labeled is calculated according to the similarity between the concept nodes obtained by the weighted concept lattice:

[0047]

[0048] Where β is the similarity weight coefficient, γ(I m , t i ) is the nearest neighbor image I m With tag t i The relationship between them, that is, if I m There exists label t i ,γ(I m , t i The value of ) is 1, otherwise it is 0.

[0049] Furthermore, to reduce irrelevant semantic labels, this invention selects label words with a support higher than 0.1 as the final label words for the image to be labeled after calculating the correlation between the weighted concepts.

[0050] Furthermore, the similarity weight coefficient β is set to 0.2.

[0051] The definitions and terms involved in this invention are as follows:

[0052] Definition 1. Formal Background: K is a triple K = (U, M, I), where U represents a set of objects, M is a set of attributes, and I represents the relationship between sets U and M. For any attribute m of an object u, it can be represented by I, which is called "object u has attribute m", denoted as uIm or (u, m)∈I.

[0053] Definition 2. A new formal background K is established by introducing weights based on the formal background K. w This formal background can be represented as a quadruple K. w = (U, M, R, W), where U is the object set, M is the attribute set, and W represents the set of single attribute weights in attribute set M, W = {w1, w2, ..., w...} n}, w i Let ∈W (1≤i≤n) represent the weight of the i-th attribute in the attribute set M. Let h w = (P, Q, w) is a quadruple K w Any triplet on, and w = W n (Q) is the weight of the attribute subset Q, which is between 0 and 1, and simultaneously satisfies the following relationship:

[0054]

[0055]

[0056] When f(P) = Q and g(Q) = P, the triple h w = (P, Q, w) satisfies the maximum expansion, at which point h w For the quadruple K wA weighted concept, P becomes the weighted concept h w The extension of Q becomes h w The connotation of.

[0057] Definition 3. In any form of background, the attribute set M = {t1, t2, ..., t...} n}, for c w = (P, Q, w), and Q = t i (1≤i≤n), then weight(Q)=weight(t i )=H(t i ) = w i w i This is called the weight of the single-attribute connotation B. For w i Normalization is performed, that is:

[0058]

[0059]

[0060] P(t i / I j ) indicates that the object is I j The time includes the attribute t i The probability of for any t i H(t) i ) indicates I j Provided to attribute t i The average amount of information.

[0061] Definition 4. If the weighted concept c w = (P, Q, w), and Q = t1∩t2∩…∩t n ,t1,t2,...,t n ∈M, weight(t) i ) = w i (1≤i≤n), then the importance weight value of the multi-label connotation Q is defined as:

[0062]

[0063] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0064] This invention utilizes information entropy to assign weights to semantic words of ancient buildings, which can objectively describe the importance of semantic tags of ancient buildings in images of ancient buildings. This is beneficial for preserving some more important connotation tags. Compared with semantic analysis by general concept lattice, the weighted concept lattice improves the accuracy of annotation after introducing tag weights, proving the effectiveness of the method. Attached Figure Description

[0065] Figure 1A flowchart of the present application;

[0066] Figure 2 A weighted concept lattice structure diagram;

[0067] Figure 3 A curve graph of the influence of different parameter r values on the labeling results;

[0068] Figure 4 A comparison graph of experimental results. DETAILED DESCRIPTION

[0069] The present application will be further described in detail below in combination with the accompanying drawings and examples.

[0070] As shown in Figure 1 and Figure 2 The ancient building image semantic completion method based on attention mechanism and weighted concept lattice includes the following steps:

[0071] S1, an initial label of an ancient building image is obtained by using a convolutional neural network with fused attention mechanism, specifically including the following steps:

[0072] S1-1, the features of the ancient building image to be labeled are extracted by an attention operator network and a VGG16 network model convolutional layer, and a weight map and a feature map with unchanged size and channel number are obtained;

[0073] The attention mechanism of the attention operator network is represented as:

[0074] f att =N(I); (1)

[0075]

[0076] In formula (1), the function N(.) is an attention operator network; I represents an input image, and the resolution of the input image is 224x224; f att represents a weight map matrix output after N(.) operation;

[0077] In formula (2), represents an attention saliency map obtained by multiplying the weight map and the feature map in position; F is a feature matrix of the input image I; represents array element multiplication in position;

[0078] The VGG16 network model has a total of 16 layers of hidden layers, all of which use the same size of convolution kernel (size of 3x3) and maximum pooling (size of 2x2), and the hidden layers include 13 convolution layers and 3 fully connected layers, and the convolution layers include 1 input layer and 1 output layer. In order to increase the amount of ancient building image data, the ancient building image data is processed by rotation, scale change, mean value, etc., and the input ancient building image is processed by scale (224x224) and gray scale normalization.

[0079] S1-2, fuse the weight map obtained in step S1-1 with the feature map to obtain an attention map capable of expressing the features of the ancient building image;

[0080] S1-3, obtain the ancient building category label through the softmax layer in the CNN structure.

[0081] S2, obtain a near neighbor label set, and calculate the weight values of each label in the near neighbor label set by using information entropy, and then construct a weighted concept lattice, specifically including the following steps:

[0082] S2-1, obtain an initial near neighbor set and an initial near neighbor label set of the image to be labeled I0;

[0083] The trained VGG16 network model is used to extract a 4096-dimensional feature vector of the image, and the Euclidean distance is used to calculate the visual similarity between the image to be labeled I0 and the training image, and the smaller the distance, the more similar the image to be labeled I0 and the training image; the visual similarity calculation formula between the image to be labeled I0 and the training image is as follows:

[0084]

[0085] In formula (3), Svis the visual similarity, dist(I i , I j ) is the Euclidean distance of the visual feature vectors of the images I i and I j ; δ is a scalar, and the value of the scalar is set as the median of the image distance matrix;

[0086] The visual similarity of the image is sorted in descending order, and the first n images with high similarity are selected as the near neighbor set and the near neighbor label set, wherein n is defined by the user; finally, the near neighbor set and the near neighbor label set are fused with the initial category label to obtain the final initial near neighbor set and the initial near neighbor label set;

[0087] S2-2, construct a weighted concept lattice:

[0088] First, the weight values of each attribute label in the initial near neighbor label set are calculated by information entropy according to the following formula (4), formula (5) and formula (6):

[0089] Attribute set M = {t1, t2, ..., t} n}, for c w = (P, Q, w), and Q = t i (1≤i≤n), then weight(Q)=weight(t i )=H(t i ) = w i and w i Normalization is performed:

[0090]

[0091]

[0092] Among them, w i p(t) represents the weight of the single attribute connotation B. i / I j ) indicates that the object is I j The time includes the attribute t i The probability of for any t i H(t) i ) indicates I j Provided to attribute t i The average amount of information;

[0093] Secondly, semantic importance analysis is performed based on the magnitude of attribute tag weights: the arithmetic mean method is used to calculate the multi-tag connotation weights, which can reflect the overall importance of the multi-tag connotations. Weights are calculated for the connotation attributes of each node in the grid structure to obtain the multi-tag weight values. If c w = (P, Q, w), and Q = t1∩t2∩…∩t n ,t1,t2,...,t n ∈M,weight(t i ) = w i (1≤i≤n), then the importance weight value of the multi-label connotation Q is defined as:

[0094]

[0095] Secondly, by setting and adjusting the importance threshold, labels that do not meet the criteria are removed;

[0096] Finally, using the nearest neighbor image set as the object and the weighted nearest neighbor label set as the formal background input, a weighted concept lattice is constructed.

[0097] S3. Perform semantic relevance analysis of ancient building images using weighted concept lattices: Based on the importance threshold of ancient building labels and concept similarity, obtain the final label set, which includes the following steps:

[0098] Firstly, the similarity sim(a, b) between any two concept nodes a and b in the weighted concept lattice is:

[0099]

[0100] Wherein, the value of sim(a, b) is between 0 and 1, 1 represents complete mapping, 0 represents error mapping; a ∨ b represents the intersection of a and b; | | represents the cardinality of the set; a LA is the lower approximation attribute set, that is, the maximum definable set containing concept a; a LA -b LA represents that the attribute set belongs to a LA , does not belong to b LA ; α is a similarity measurement threshold, the value is set to 0.5, when α ≥ 0.5, there is a mapping relationship between the concept nodes a and b, when α < 0.5, there is no association between the concept nodes a and b;

[0101] Secondly, the visual similarity between the image to be labeled and the nearest neighbor image is taken as the support degree of the nearest neighbor image I m to the image to be labeled, and the support degree of each label in the candidate label set to the image to be labeled is calculated according to the similarity between the concept nodes obtained by the weighted concept lattice:

[0102]

[0103] Wherein, β is a similarity weight coefficient, the value of β is 0.2; γ(I m , t i ) is the belonging relationship between the nearest neighbor image I m and the label t i , that is, if I m has the label t i , the value of γ(I m , t i ) is 1, otherwise 0. In order to reduce irrelevant semantic labels, the application selects the label word with a support degree higher than 0.1 as the final label word of the image to be labeled after calculating the association degree between the weighted concepts.

[0104] In this specific embodiment, images of ancient buildings from the Tang and Song dynasties were collected online, constructing a dataset of images of ancient buildings from the Tang and Song dynasties. To verify the effectiveness of the invention, experiments were conducted on the dataset of ancient buildings from the Tang and Song dynasties and the traditional ancient building dataset provided by the State Key Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences (http: / / visi on.ia.ac.cn / data). Based on the characteristics of ancient Chinese architecture, semantic labels were set from aspects such as roof style, building components, and roof type. There are five main roof styles in ancient architecture: Wudian roof, Xieshan roof, Xuanshan roof, Yingshan roof, and Cuanjian roof; the main building components are wood structure and brick structure; and the eaves are divided into single-eave and double-eave forms.

[0105] The attribute tables for the Tang and Song Dynasty dataset and the traditional ancient architecture dataset in this specific embodiment are shown in Table 1 below.

[0106] Table 1

[0107]

[0108] Parametric analysis of parameter r was conducted through experiments on two types of ancient building datasets (e.g., ...). Figure 3 (As shown in the figure), r represents the intrinsic importance threshold. The F-value is used to evaluate this parameter to verify the rationality of the threshold r. Figure 3 The results are obtained from the analysis of parameter r. As can be seen from the figure, the evaluation index curve fluctuates to some extent with the change of r value, showing a trend of first rising and then falling. The labeling performance reaches its best when the r value reaches 0.15, indicating that setting the connotation importance threshold to 0.15 is reasonable and helps to distinguish tags and keywords of different importance. The formal background K is shown in Table 2 below.

[0109] Table 2

[0110]

[0111] This invention conducted a comparative annotation experiment on two types of ancient building datasets on the Corel 5K dataset, and the parameters are shown in Table 3 below.

[0112] Table 3

[0113]

[0114] The three experiments are repeated, and the average values of P, R and F are taken. In Table 3, VAM represents the visual attention method used in the experiment, FCA and WFCA represent the general concept lattice and the weighted concept lattice respectively. The comparison results are shown in Table 3 Figure 4 As shown in Table 3, compared with the basic VGG16 method, the accuracy P and the recall rate of the weighted concept lattice method of the present application are greatly improved, reaching 47% and 41% respectively. This is because the visual attention mechanism used in the present application enhances the significant features of the image and improves the accuracy of image labeling. Since the ancient building near image set is obtained by searching the ancient building image bottom features in descending order, and the weighted concept lattice semantic relevance analyzes the similarity between semantics, the labeling result takes into account both the bottom features and the high-level semantics of the ancient building image, thereby effectively avoiding the omission of important semantics and improving the recall rate of labeling. Compared with the general concept lattice, although the recall rate decreases, the accuracy is improved by 2%, which can prove that the present application can effectively improve the accuracy of ancient building image labeling and further prove the effectiveness of the weighted concept lattice semantic analysis.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for semantic completion of ancient building images based on attention mechanism and weighted concept lattice, characterized in that, The method comprises the following steps: S1, obtaining initial labels of ancient building images by using a convolutional neural network with a fusion attention mechanism; S2, obtaining a near neighbor label set and calculating the weight values of each label in the set by using information entropy, and then constructing a weighted concept lattice; the step S2 comprises the following steps: S2-1, obtaining an initial near neighbor set and an initial near neighbor label set of the image I0 to be labeled; S2-2, constructing a weighted concept lattice: first, calculating the weight values of each attribute label in the initial near neighbor label set by using information entropy; second, performing semantic importance analysis according to the size of the attribute label weight values; third, removing labels that do not meet the conditions by setting and adjusting the connotation importance threshold; finally, taking the near neighbor image set as an object and taking the near neighbor label set with weight values as a formal context input, and constructing a weighted concept lattice; S3, performing semantic correlation analysis of ancient building images by using the weighted concept lattice: obtaining a final label set according to the connotation importance threshold of the ancient building label and the concept similarity; the step S3 comprises the following steps: First, the similarity sim(a, b) between any two concept nodes a and b in the weighted concept lattice is: Wherein, the value of sim(a, b) is between 0 and 1, 1 represents complete mapping, 0 represents error mapping; a represents the intersection of a and b; || represents the cardinality of the set; a LA is the lower approximate attribute set, that is, the maximum definable set containing the concept a; a LA -b LA represents that the attribute set belongs to a LA , does not belong to b LA ; alpha is a similarity measure threshold, the value is set to 0.5, when alpha >= 0.5, there is a mapping relationship between the concept nodes a and b, when alpha < 0.5, there is no association between the concept nodes a and b; Secondly, the visual similarity between the image to be labeled and the near neighbor image is taken as the near neighbor image I m The support degree of each label in the candidate label set to the image to be labeled is calculated according to the similarity between the concept nodes obtained from the weighted concept lattice: where β is a similarity weight coefficient, γ(I m ,t i ) is the membership relation between the neighbor image I m and the label t i , i.e. γ(I m ,t i ) is 1 if I m has the label t i , otherwise 0.

2. The ancient building image semantic completion method based on attention mechanism and weighted concept lattice according to claim 1, characterized in that: The step S1 comprises the following steps: S1-1, extracting features of the ancient building image to be labeled by using an attention operator network and a VGG16 network model convolution layer to obtain a weight map and a feature map with unchanged size and channel number; S1-2, fusing the weight map and the feature map obtained in step S1-1 to obtain an attention map capable of expressing the features of the ancient building image; S1-3, obtaining an ancient building category label by using a softmax layer in a CNN structure.

3. The ancient building image semantic completion method based on attention mechanism and weighted concept lattice according to claim 1 or 2, characterized in that: The attention mechanism of the attention operator network is represented as: f att = N(I); (1) In formula (1), the N(.) function is an attention operator network; I represents an input image, and the input image resolution is 224x224; f att represents a weight map matrix output after the N(.) operation; In formula (2), represents the attention saliency map obtained by multiplying the weight map and the feature map; F is a feature matrix of the input image I; represents the array element pair multiplication.

4. The ancient building image semantic completion method based on attention mechanism and weighted concept lattice according to claim 2, characterized in that: The VGG16 network model has a total of 16 hidden layers, all of which use the same size of convolution kernel and maximum pooling, and the hidden layers include 13 convolution layers and 3 fully connected layers, and the convolution layers include 1 input layer and 1 output layer.

5. The ancient building image semantic completion method based on attention mechanism and weighted concept lattice according to claim 4, characterized in that: The size of the convolution kernel is 3x3, and the size of the maximum pooling is 2x2.

6. The ancient building image semantic completion method based on attention mechanism and weighted concept lattice according to claim 1, characterized in that: The step S2 comprises the following steps: S2-1, obtaining an initial near neighbor set and an initial near neighbor label set of the image I0 to be labeled; Extracting a 4096-dimensional feature vector of the image by using the trained VGG16 network model, and calculating the visual similarity of the image I0 to be labeled and the training image by using the Euclidean distance, wherein the smaller the distance, the more similar the image I0 to be labeled and the training image; the visual similarity calculation formula of the image I0 to be labeled and the training image is as follows: In formula (3), Sv is a visual similarity, dist(I i , I j ) is a Euclidean distance of visual feature vectors of images I i and I j ; and δ is a scalar, and the value of the scalar is set as a median of the image distance matrix. The visual similarity of the image is sorted in descending order, and the top n images with high similarity are selected as the near neighbor set and the near neighbor label set, wherein n is defined by the user; finally, the near neighbor set and the near neighbor label set are fused with the initial category label to obtain the final initial near neighbor set and the initial near neighbor label set; S2-2, constructing a weighted concept lattice: First, calculating the weight values of each attribute label in the initial near neighbor label set by using information entropy: M = {t1, t2,..., tn}, for c = (P, Q, w), and Q = t n (1≤i≤n), weight(Q) = weight(t w ) = H(t i ) = w i , and w i is normalized. i i}, for c = (P, Q, w), and Q = t​ where w i is the weight of the single attribute connotation B, p(t i / I j ) represents the probability that the object contains attribute t j when the object is I i , H(t i / I i ) represents the average amount of information provided by attribute t j to I i ; Secondly, according to the size of the attribute label weight, the semantic importance analysis is carried out: if c w =(P,Q,w), and Q=t1∩t2∩…∩t n n, t1,t2,...,t n n∈M, weight(t i )i=w i (1≤i≤n), then the importance weight value of the multi-label connotation Q is defined as: Third, removing labels that do not meet the conditions by setting and adjusting the connotation importance threshold; Finally, taking the near neighbor image set as an object and taking the near neighbor label set with weight values as a formal context input, and constructing a weighted concept lattice.

7. The ancient building image semantic completion method based on attention mechanism and weighted concept lattice according to claim 1, characterized in that: Select the label word with support higher than 0.1 as the final label word of the image to be labeled.

8. The ancient building image semantic completion method based on attention mechanism and weighted concept lattice according to claim 1, characterized in that: The value of the similarity weight coefficient β is 0.

2. The value of the similarity weight coefficient β is 0.2.

Images