A method, device, equipment and medium for identifying historical architectural styles

Through U-Net and generative adversarial networks, the mixed style feature maps of historical buildings are extracted and marked, and the problem of marking deviation of mixed style images is solved, and the accurate recognition and positioning of historical building styles is achieved, thereby improving the recognition accuracy.

CN118072164BActive Publication Date: 2024-11-22INNER MONGOLIA UNIV OF TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410180935.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-18
Publication Date
2024-11-22
Estimated Expiration
2044-02-18

AI Technical Summary

Technical Problem

In the prior art, the mixed style image marking deviation of historical buildings leads to low accuracy in model recognition, making it difficult to accurately capture the style characteristics of the buildings.

Method used

U-Net network and generative adversarial network are adopted to extract the style feature maps of single-style buildings, build a mixed style feature map, and use a decoder to mark it, eliminating subjective view bias and achieving accurate recognition of mixed style images.

Benefits of technology

It improves the accuracy of historical building style recognition, can accurately locate and identify different style elements, eliminate marking deviations, and enhances the recognition ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118072164B_ABST
    Figure CN118072164B_ABST
Patent Text Reader

Abstract

The present invention provides a method, device, equipment and medium for identifying the style of historical buildings. The method includes collecting mixed-style historical building images, marking the mixed-style historical building images, and constructing a data set; establishing a neural network for identifying the style of historical buildings and training it; and using the trained neural network to perform style recognition on the mixed-style historical building images to be identified. The present invention can capture the unique features of historical buildings of different styles by extracting a style feature map of a single-style historical building image, using an encoder of a U-Net network to obtain a mixed feature map, finding and marking the mixed feature map that matches the style feature map, and accurately locating and identifying different style elements in the mixed-style historical building image. Decoding based on the marked feature map can accurately reproduce the marks of different styles in the mixed-style image, eliminating the deviation of subjective views on the marks of the same image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of historical building style recognition, and in particular to a historical building style recognition method, device, equipment and medium. Background Art

[0002] Historical architectural style identification is a process of classifying historical buildings into different styles or periods by analyzing their appearance, structure, and design features. Identifying historical architectural styles can have a variety of uses, including architectural cultural heritage protection, historical architectural style evaluation, protection and utilization of historical and cultural blocks, virtual tourism and experience, and development of historical architectural cultural tourism resources. Automated historical architectural style identification can more effectively manage and protect cultural heritage, help researchers understand the history of historical architectural development, and also create more realistic historical scenes in virtual environments. Traditionally, the identification of historical architectural styles is done by architectural professionals through their knowledge of their appearance, materials, structure, etc.

[0003] At present, with the development of computer vision and machine learning, the use of algorithms and models for automatic historical architectural style recognition has become more and more common. The field of visual recognition has shifted from traditional recognition based on feature engineering such as SVM algorithms to recognition based on neural network model architecture. The general steps of learning features from historical building images and identifying the style of historical buildings based on neural network model architecture are: collecting different historical building image data and marking these historical building images with styles, selecting appropriate neural network models such as LeNet, AlexNet, VGG, ResNet, etc., inputting the marked data into the network for training, and making the network learn the characteristics of different architectural styles during the training process. Finally, the trained network is used to perform style recognition on historical building images of unknown styles. However, in the above steps, if the style characteristics of the unknown style historical building images are obvious, such as Notre Dame de Paris has a strong Gothic architectural style and the Roman Jesuit Church has a strong Baroque architectural style, the style recognition results are often accurate. However, historical buildings are like precious traces left by the years, telling the evolution of society and the interaction between civilizations in their own way, and witnessing political changes, cultural integration and technological development. In this long and complex historical process, many historical buildings were not built in a short time. During their construction, they were directly or indirectly influenced by different times, cultures, and craftsmen. They should reflect different architectural styles and diverse cultural characteristics. As a result, the styles of historical buildings have become more complex and diverse. Many buildings are no longer limited to specific traditional styles, but adopt a combination of multiple elements to create a unique and rich appearance. Therefore, a single historical architectural style recognition method often cannot cope with this diversity and complexity well, because when different parts of historical buildings adopt different styles, the traditional overall architectural style recognition method is powerless. In this case, the neural network may be affected by the mixed style, making it difficult to accurately capture the style characteristics of the historical building. In order to overcome this problem, researchers usually use the method of historical building image segmentation to divide the image into different regions and manually mark each region with a separate style.

[0004] However, when manually labeling historical building images, the subjective opinions of different labelers may lead to deviations in the labeling of the same image, causing the model to learn inconsistent feature representations during the training process, thereby reducing the recognition accuracy of the model. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for identifying historical building styles, which can solve the technical problem in the prior art that the recognition accuracy of the model is not high due to the marking deviation of mixed-style historical building images.

[0006] The present invention provides a method for identifying historical architectural styles, comprising:

[0007] Collect images of historical buildings with mixed styles, label them, and construct a data set;

[0008] Establish a neural network for identifying historical architectural styles, input the data set into the neural network for training; use the trained neural network to perform style recognition on the mixed-style historical architectural images to be identified;

[0009] The process of marking mixed-style historical building images includes the following steps:

[0010] Collecting images of historical buildings in a single style, and extracting style feature maps of the images of historical buildings in a single style;

[0011] Construct a U-Net network, and use the encoder in the U-Net network to obtain a mixed feature map of mixed-style historical building images;

[0012] Marking the mixed feature map that matches the style feature map;

[0013] The decoder in the U-Net network is used to decode the labeled mixed feature map to obtain labels of different styles in the mixed-style historical building images.

[0014] Furthermore, the extracting of a style feature map of a historical building image of a single style includes:

[0015] Collect n different styles of single-style historical building images;

[0016] Construct a generative adversarial network including a generator G and a discriminator D, input n different styles of single-style historical building images into the generative adversarial network for training, and obtain n trained generative adversarial networks;

[0017] Extract n generators G from n trained generative adversarial networks, and use n generators G to generate n style feature maps.

[0018] Furthermore, the U-Net network includes:

[0019] An encoder for extracting high-level features of an input image;

[0020] A decoder for mapping high-level features back to the spatial structure of the input image;

[0021] A skip connection is used to connect each layer of the encoder to the corresponding layer of the decoder.

[0022] Furthermore, the method of using the encoder in the U-Net network to obtain a mixed feature map of the mixed-style historical building image includes:

[0023] Input the mixed-style historical building images into the U-Net network that obtains initial weights based on transfer learning;

[0024] The first encoder layer in the encoder extracts the underlying feature maps such as edges and textures in the mixed-style historical building images;

[0025] The middle encoder layer in the encoder obtains the abstract semantic information feature map by reducing the size of the underlying feature map and increasing the number of channels of the underlying feature map;

[0026] The final encoder layer obtains a multi-channel mixed feature map of mixed-style historical building images based on the abstract semantic information feature map.

[0027] Furthermore, marking the mixed feature map that matches the style feature map includes:

[0028] Calculate the cosine similarity between each channel mixed feature map and n style feature maps respectively;

[0029] The mixed feature map of each channel is marked as the style corresponding to the style feature map with the smallest cosine similarity.

[0030] Furthermore, the decoder in the U-Net network is used to decode the marked mixed feature map, including:

[0031] Each layer decoder takes the feature map output by the previous layer decoder and the feature map of the corresponding encoder layer as input, performs upsampling operation on the input to gradually restore the spatial information of the image; among which, the input of the bottom layer decoder is a mixed feature map with labels;

[0032] Get the output image of the last layer decoder, which contains markers of different styles.

[0033] Furthermore, the step of inputting the data set into the neural network for training includes:

[0034] Construct the yolov5s neural network and randomly initialize the weights of the yolov5s neural network;

[0035] The training set in the dataset is input into the yolov5s neural network, and the predicted style of each historical building structure in the image is obtained based on the initialized weights;

[0036] Set the cross entropy loss function, and use the loss function to calculate the gap between the predicted style and the actual style of the yolov5s neural network; back-propagate the gap, and use the ADAM optimization algorithm to update the weights of the yolov5s neural network to minimize the gap between the predicted style and the actual style;

[0037] When the loss function converges, the trained yolov5s neural network is obtained.

[0038] A device for identifying historical architectural styles, comprising:

[0039] A data acquisition module, used to collect images of historical buildings of mixed styles;

[0040] Data annotation module, used for:

[0041] Collecting images of historical buildings in a single style, and extracting style feature maps of the images of historical buildings in a single style;

[0042] Construct a U-Net network, and use the encoder in the U-Net network to obtain a mixed feature map of mixed-style historical building images;

[0043] Marking the mixed feature map that matches the style feature map;

[0044] Use the decoder in the U-Net network to decode the labeled mixed feature map to obtain labels of different styles in the mixed-style historical building image;

[0045] Style recognition module, used to:

[0046] A neural network for identifying historical architectural styles is established, and the data set is input into the neural network for training; the trained neural network is used to perform style recognition on mixed-style historical architectural images to be identified.

[0047] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying the historical architectural style is implemented.

[0048] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned historical architectural style recognition method when executing the program.

[0049] The present invention provides a method, device, equipment and medium for identifying historical architectural styles. Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention can capture the unique features of historical buildings of different styles by extracting the style feature map of a historical building image of a single style, obtain a mixed feature map using the encoder of the U-Net network, find and mark the mixed feature map that matches the style feature map, accurately locate and identify the different style elements in the mixed-style historical building image, decode based on the marked feature map, accurately reproduce the marks of different styles in the mixed-style image, and eliminate the deviation of subjective views on the marks of the same image. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0052] In the attached picture:

[0053] Figure 1 It is a flow chart provided in this manual;

[0054] Figure 2 This is the U-Net network structure diagram provided in this manual;

[0055] Figure 3 It is a schematic diagram of the structure of the generative adversarial network provided in this manual;

[0056] Figure 4 It is a symbol provided in this manual;

[0057] Figure 5 It is a schematic diagram of the computer equipment provided in this manual. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention, but it should be understood that the protection scope of the present invention is not limited by the specific implementation methods.

[0059] Example

[0060] like Figure 1 As shown, a method for identifying the style of historical buildings includes the following steps: collecting mixed-style historical building images; marking the mixed-style historical building images to construct a data set; establishing a neural network for identifying the style of historical buildings, inputting the data set into the neural network for training; and using the trained neural network to perform style recognition on the mixed-style historical building images to be identified. Specifically, it includes:

[0061] Step 1: Collect mixed-style historical building images:

[0062] Identify the types and characteristics of historical building styles, which will help guide the direction of image collection and ensure that the collected images cover a variety of styles. Based on the definition of style, select geographical locations that contain historical buildings of multiple styles. Choose appropriate photographic equipment to collect images of historical buildings: collect images in different seasons and weather conditions to obtain a more diverse and rich appearance of historical buildings; shoot from different angles and distances to obtain a more comprehensive set of images.

[0063] The collected mixed-style historical building images are marked. The process of marking the mixed-style historical building images in this manual is: first, use the generator of the generative adversarial network to generate style feature maps corresponding to various styles; then input the mixed-style historical building images into the U-Net network to obtain the mixed feature map of the mixed-style historical building images based on the encoder; add a mark to the mixed feature map by calculating the similarity between the mixed feature map and the feature maps of different styles; pass the mixed feature map with the mark through the decoder of the U-Net network to obtain the final mixed-style historical building image with the mark. The specific steps include:

[0064] Step 2: Use generative adversarial networks to extract style feature maps

[0065] Collect a single-style historical building image. This manual uses a generative adversarial network (GAN) to extract a style feature map of a single-style historical building image. By using the generator of the generative adversarial network to generate style feature maps corresponding to various styles, rich and diverse style representations can be obtained, which helps to identify and mark different style elements in mixed-style images, so that the system can more comprehensively understand and capture various artistic styles, architectural styles and other features in the image. The specific steps include:

[0066] Step 2.1. Data preparation

[0067] Collect a dataset of historical building images of a single style in multiple different styles to ensure that the dataset contains sufficient diversity and representativeness.

[0068] The image is preprocessed, including resizing, normalization, etc., so that it can be input into the generative adversarial network.

[0069] Step 2.2: Build a Generative Adversarial Network:

[0070] like Figure 2 As shown, the generative adversarial network includes a generator and a discriminator, wherein the generator includes:

[0071] Input layer, whose input is uniformly distributed noise.

[0072] The fully connected layer is used to convert the input uniformly distributed noise into a two-dimensional tensor.

[0073] The UpSampling layer is used to upsample the output of the previous layer.

[0074] Convolutional layer, used to generate feature maps.

[0075] Batch Normalization layers are used to stabilize the training process by normalizing the input to each unit.

[0076] The output layer is used to output the generated pseudo building image.

[0077] The discriminator includes:

[0078] Several modules consisting of convolutional layers, pooling layers, and non-linear mapping layers.

[0079] The output layer composed of the Sigmoid function is used to output the judgment output result of the discriminant model D on whether the input image is a real building image:

[0080] If the discriminant model D determines that the input image is a real building image, its output is 1; if the discriminant model D determines that the input image is a fake building image, its output is 0.

[0081] Step 2.3: Train Generative Adversarial Network

[0082] Randomly set the initial weights of the generator and discriminator.

[0083] Uniformly distributed noise is input into the generator, and pseudo building images similar to real building images are generated based on the initial weights of the generator.

[0084] The pseudo-building image and the real building image are input into the discriminator, the input image is discriminated based on the initial weight of the discriminator, and the discrimination result is output.

[0085] Loss function definition: Define the loss functions of the generator and discriminator, usually using adversarial loss.

[0086] The weight parameters of the generator and discriminator are updated by using the stochastic gradient ascent optimization algorithm to minimize the value of the loss function. Multiple rounds of training are performed by alternately optimizing the generator and discriminator. When the loss function converges, the trained generative adversarial network is obtained.

[0087] Input n different styles of single-style historical building images to the discriminator, and obtain n generators, each of which can generate feature maps of the corresponding style. At this time, the feature map generated by the convolutional layer of the generator can be regarded as the generator's abstract representation of the input noise, where each feature map corresponds to a different historical building style feature.

[0088] Step 3: Build U-Net network

[0089] The U-Net network is a convolutional neural network used for image segmentation tasks, such as Figure 3 As shown in the figure, it includes: Encoder, which is mainly responsible for extracting high-level features of the input image, gradually reducing the spatial resolution through convolutional layers and pooling layers, and extracting semantic information at different levels; intermediate feature extraction, which retains the intermediate features of the input image and helps to restore detail information in the decoder; Decoder, which is responsible for mapping the high-level abstract representation back to the spatial structure of the input image, and gradually restores the spatial resolution through upsampling layers and convolutional layers, and skip connections, which connect each layer of the encoder to the corresponding layer of the decoder to fuse the bottom and high-level information and help the network better capture local and global features; the final classification layer, the last layer of the decoder, is usually a convolutional layer, which outputs the classification result of the model for each pixel to form a building segmentation mask.

[0090] Among them, by calculating the similarity between the mixed feature map and the different style feature maps, it is possible to more accurately determine which parts of the mixed style image correspond to different styles. This helps to improve the accuracy of the labeling, so that the system can more accurately locate and identify different style elements, providing a more reliable basis for subsequent processing. Then, by using the decoder of the U-Net network, the mixed feature map with labels can be decoded into the final mixed style building image. The application of the decoder enables the system to convert the labeled information into a visual image, completing the restoration and reconstruction of the mixed style image. This helps to ultimately generate a more artistic and visually attractive mixed style historical building image. The U-Net network constructed in this specification is based on transfer learning. Transfer learning is a method of applying knowledge learned from one task to another related task. The benefit of transfer learning is that it utilizes the features and knowledge learned from one task, thereby accelerating the convergence of the model and improving performance on similar tasks. Transfer learning is performed in the U-Net network, especially in tasks such as image segmentation. This method can usually effectively improve the generalization ability of the model. The steps for transfer learning of U-Net networks are usually as follows: select a U-Net model trained on a large-scale dataset in the field of image segmentation as the basis; freeze the bottom layer of the pre-trained model, that is, the layer close to the input, to retain the information learned on the low-level features; modify the output layer of the network according to the needs of the new task; select the network layer that needs to be fine-tuned according to the similarity of the task and the size of the dataset. You can choose to unfreeze some of the top convolutional layers so that the network can better adapt to the new task; select an appropriate learning rate for the transfer learning process. Generally speaking, a smaller learning rate can be used for the weights of the underlying network, while a larger learning rate can be used for the newly added layers; fine-tune the entire network using the new task dataset. This stage may require fewer iterations because the bottom layer already contains common features learned from the pre-training task.

[0091] Step 4: Use the encoder in the U-Net network to obtain the mixed feature map of the mixed style building image

[0092] This manual inputs a mixed-style historical building image into the U-Net network, obtains a mixed feature map based on the encoder, and labels the mixed feature map by calculating the similarity between the mixed feature map and the feature maps of different styles. This step enables the system to accurately identify various style elements in the mixed-style image, providing key information for subsequent decoding and reconstruction. The encoder in the U-Net includes:

[0093] First encoder layer: The first encoder layer usually captures some low-level features, such as edges, textures, colors, etc. These low-level features are the initial representation of the image and can help the network perceive the details and local structures of the image.

[0094] Intermediate encoder layers: In the encoder part of the U-Net network, the intermediate encoder layers gradually reduce the size of the feature map but increase the number of channels of the feature map. These layers capture higher-level features of the image, such as texture patterns, object shapes, and partial structures. These feature representations can express more abstract semantic information in the image.

[0095] Last encoder layer: The feature maps of the last encoder layer usually contain the global semantic information of the image. These feature maps reflect important properties such as the overall shape, location, and presence of objects in the image. In the last encoder layer, the size of the feature maps is smaller, but the number of channels is larger, providing a rich global image representation.

[0096] Through the various layers of the encoder, the U-Net network can extract different levels of semantic features from the image, including:

[0097] The lowest level feature map: The lowest level of the encoder usually downsamples the input image multiple times to obtain smaller but more semantically informative feature maps. These feature maps have lower spatial resolution but contain global contextual information.

[0098] Intermediate layer feature maps: As the encoder goes deeper, the size of the feature maps decreases. These intermediate layer feature maps contain more abstract and local features, which can capture the medium-scale structure and texture information in the image.

[0099] Highest-level feature map: The highest-level feature map of the encoder usually has the smallest spatial resolution, but contains the most abstract and advanced features. These features are crucial for the overall semantic segmentation task. This manual uses the multi-channel highest-level feature map output by the last encoder layer as the mixed feature map of the mixed-style historical building image.

[0100] Through this hierarchical feature map design, U-Net captures semantic information of different scales and levels in the encoder. As the feature map gradually increases in the network level, the level of abstraction of information becomes higher and higher, enabling the network to understand the global structure and local details of the image, thereby better performing semantic segmentation tasks. This pyramid-like feature map structure helps improve the model's ability to perceive information at all scales of the image. The decoder part uses these feature maps to gradually restore the segmentation results and reconstruct the semantic information to achieve the goal of image segmentation.

[0101] Step 5: Add markers to the mixed feature map

[0102] According to step 4, a multi-channel mixed feature map can be obtained, and the gap between each channel map and the n style feature maps is calculated respectively, and each element in the feature map of each channel is marked as the style mark corresponding to the style feature map with the smallest gap therewith.

[0103] like Figure 4 As shown, for each channel feature map obtained, the difference or distance between the channel map and these reference style feature maps is calculated respectively. This can be achieved by calculating the cosine similarity s ij (i is the i-th channel feature map, j is the j-th style feature map) to measure the difference between features.

[0104] For each pixel or element in each channel, after calculating the difference with n style feature maps, the style feature map with the smallest difference (smallest cosine similarity) is selected, and the pixel is marked as the style tag corresponding to the style feature map with the smallest difference. The purpose of the whole process is to assign a "style tag" to each pixel in each channel of the feature map, which represents the reference style pattern with the smallest difference with the pixel.

[0105] Step 6: Use the decoder of the U-Net network to decode the mixed feature map with labels

[0106] The decoder plays a key role in the U-Net network. Once the mixed-style historical building images are successfully marked, the next step is to decode these marks into historical building features of different styles. In the U-Net network, the encoder gradually reduces the spatial resolution of the image and extracts high-level abstract features. The decoder gradually restores the spatial information of the image through upsampling operations. Each decoder layer upsamples the low-resolution feature map to a higher resolution to better restore the details and spatial structure of the image. After the multi-channel feature map is obtained in the last layer of the U-Net network encoder, the pixels of one of the feature maps are marked, and then tracking these marked pixels during the decoding process can be achieved by the following steps:

[0107] After labeling specific channels in the feature map, create a labeled image with the same size as the original image. In the labeled image, set the labeled pixels to specific values ​​or colors for tracking during the decoding process.

[0108] During the decoding process, each decoder layer of the decoder performs feature fusion based on the feature map of the corresponding encoder layer. In order to track the labeled pixels, the labeled image needs to be aligned with the feature map used in the decoding process to ensure that they have the same spatial resolution and size. In each decoder layer of the decoder, the labeled image is combined with the corresponding feature map before or after feature fusion. The labeled image can be applied to the feature map using pixel-level operations such as element-wise multiplication or masking operations, which can preserve the feature information corresponding to the labeled pixels. The labeled image is passed between different decoder layers. The resolution of the labeled image can be adjusted to match the size of the decoder layer feature map through operations such as upsampling or downsampling. If upsampling or downsampling changes the position of the pixels, interpolation methods can be used to reallocate the labels. By propagating the labeled image layer by layer during the decoding process, it is ensured that the labeled pixels are tracked between different decoding layers. In each decoding layer, the resolution of the labeled image is adjusted as needed to ensure that it matches the size of the decoding layer feature map.

[0109] Finally, the labeled image is combined with the last layer of feature maps of the decoder to obtain the decoded image pixel values ​​corresponding to the labeled pixels. At this point, the labels can be applied to the decoded results using masking, weighted fusion, or other pixel-level processing methods.

[0110] Step 7: Build a dataset based on the decoded historical building images and train the recognition model

[0111] Construct the yolov5s neural network and randomly initialize the weights of the yolov5s neural network;

[0112] The training set in the dataset is input into the yolov5s neural network, and the predicted style of each historical building component in the image is obtained based on the initialized weights;

[0113] Set the cross entropy loss function and use the loss function to calculate the difference between the predicted style and the actual style of the yolov5s neural network;

[0114] The gap is back-propagated, and the ADAM optimization algorithm updates the weights of the yolov5s neural network to minimize the gap between the predicted style and the actual style;

[0115] When the loss function converges, the trained yolov5s neural network is obtained.

[0116] The specific definition of the historical architectural style recognition device can be found in the definition of the historical architectural style recognition method above, which will not be repeated here. Each module in the above-mentioned historical architectural style recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0117] The present specification also provides a computer-readable storage medium, which stores a computer program, and the computer program can be used to execute the above-mentioned historical architectural style identification method.

[0118] This manual also provides Figure 5 The structural diagram of the computer device shown in FIG. Figure 5 As shown, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned historical architectural style recognition method.

[0119] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0120] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for identifying historical architectural styles, characterized in that: include: Collect images of historical buildings with mixed styles, label them, and construct a data set; Establish a neural network for identifying historical architectural styles and input the data set into the neural network for training; Use the trained neural network to perform style recognition on the mixed-style historical building images to be identified; The process of marking mixed-style historical building images includes the following steps: Collecting a single-style historical building image and extracting a style feature map of the single-style historical building image; including: Collect n different styles of single-style historical building images; Construct a generative adversarial network including a generator G and a discriminator D, input n different styles of single-style historical building images into the generative adversarial network for training, and obtain n trained generative adversarial networks; Extract n generators G from n trained generative adversarial networks, and use n generators G to generate n style feature maps; Construct a U-Net network and use the encoder in the U-Net network to obtain a mixed feature map of mixed-style historical building images; including: Input the mixed-style historical building images into the U-Net network that obtains initial weights based on transfer learning; The first encoder layer in the encoder extracts the underlying feature maps of edges and textures in the mixed-style historical building images; The middle encoder layer in the encoder obtains the abstract semantic information feature map by reducing the size of the underlying feature map and increasing the number of channels of the underlying feature map; The final encoder layer obtains a mixed feature map of multiple channels of mixed-style historical building images based on the abstract semantic information feature map; Marking the mixed feature map that matches the style feature map; including: Calculate the cosine similarity between each channel mixed feature map and n style feature maps respectively; Mark the mixed feature map of each channel as the style corresponding to the style feature map with the smallest cosine similarity; The labeled mixed feature map is decoded using the decoder in the U-Net network, including: Each layer decoder takes the feature map output by the previous layer decoder and the feature map of the corresponding encoder layer as input, performs upsampling operation on the input to gradually restore the spatial information of the image; among which, the input of the bottom layer decoder is a mixed feature map with labels; Get the output image of the last layer decoder, which contains markers of different styles; Obtaining labels of different styles in mixed-style historical building images.

2. The architectural style recognition method according to claim 1, characterized in that: The U-Net network includes: An encoder for extracting high-level features of an input image; A decoder for mapping high-level features back to the spatial structure of the input image; A skip connection is used to connect each layer of the encoder to the corresponding layer of the decoder.

3. The method for identifying historical architectural styles according to claim 1, characterized in that: The step of inputting the data set into the neural network for training includes: Construct the yolov5s neural network and randomly initialize the weights of the yolov5s neural network; The training set in the dataset is input into the yolov5s neural network, and the predicted style of each historical building structure in the image is obtained based on the initialized weights; Set the cross entropy loss function, and use the loss function to calculate the gap between the predicted style and the actual style of the yolov5s neural network; back-propagate the gap, and use the ADAM optimization algorithm to update the weights of the yolov5s neural network to minimize the gap between the predicted style and the actual style; When the loss function converges, the trained yolov5s neural network is obtained.

4. A device for identifying historical architectural styles, characterized in that: include: A data acquisition module, used to collect images of historical buildings of mixed styles; Data annotation module, used for: Label images of historical buildings of mixed styles and construct a dataset; The process of marking mixed-style historical building images includes the following steps: Collecting a single-style historical building image and extracting a style feature map of the single-style historical building image; including: Collect n different styles of single-style historical building images; Construct a generative adversarial network including a generator G and a discriminator D, input n different styles of single-style historical building images into the generative adversarial network for training, and obtain n trained generative adversarial networks; Extract n generators G from n trained generative adversarial networks, and use n generators G to generate n style feature maps; Construct a U-Net network and use the encoder in the U-Net network to obtain a mixed feature map of mixed-style historical building images; including: Input the mixed-style historical building images into the U-Net network that obtains initial weights based on transfer learning; The first encoder layer in the encoder extracts the underlying feature maps of edges and textures in the mixed-style historical building images; The middle encoder layer in the encoder obtains the abstract semantic information feature map by reducing the size of the underlying feature map and increasing the number of channels of the underlying feature map; The final encoder layer obtains a mixed feature map of multiple channels of mixed-style historical building images based on the abstract semantic information feature map; Marking the mixed feature map that matches the style feature map; including: Calculate the cosine similarity between each channel mixed feature map and n style feature maps respectively; Mark the mixed feature map of each channel as the style corresponding to the style feature map with the smallest cosine similarity; The labeled mixed feature map is decoded using the decoder in the U-Net network, including: Each layer decoder takes the feature map output by the previous layer decoder and the feature map of the corresponding encoder layer as input, performs upsampling operation on the input to gradually restore the spatial information of the image; among which, the input of the bottom layer decoder is a mixed feature map with labels; Get the output image of the last layer decoder, which contains markers of different styles; Obtaining labels of different styles in images of mixed-style historical buildings; Style recognition module, used to: A neural network for identifying historical architectural styles is established, and the data set is input into the neural network for training; the trained neural network is used to perform style recognition on mixed-style historical architectural images to be identified.

5. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

6. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 3 is implemented.