Intelligent recognition system for urban and rural building styles and features based on multi-source data fusion

Abstract

The invention discloses an intelligent recognition system for urban and rural building styles and features based on multi-source data fusion. According to the invention, by fusing the emotion feedback and objective visual features of the user on the building, the system not only can accurately judge the style type of the building, but also can understand the subjective feeling difference caused by different styles. For example, when the Hui-style building is identified, the system can synchronously capture the worry emotion generated by the user on the'whitewashing and yerba ', so that the characteristics of the building can be displayed and the emotion demand of the tourist can be matched during tourism recommendation. The dynamic learning mechanism enables the system to continuously absorb fresh feedback of users in various regions and automatically correct cognitive deviation of an original model to regional characteristic buildings, for example, the detail distinguishing standard of southern Fujian red brick placing and Lingnan wok ear house is updated in time, the adaptive capacity to continuously changing urban and rural styles is kept, and in a cultural heritage protection scene, the system can be used for protecting cultural heritage. In combination with double dimensions of professional surveying and mapping data and public emotion evaluation, recent modern buildings with collective memory value can be positioned more accurately.

Description

Technical Field

[0001] The present invention belongs to the technical field of urban and rural building recognition, and specifically provides an intelligent recognition system for urban and rural building features based on multi-source data fusion. Background Art

[0002] Urban and rural architectural style identification refers to the process of systematically investigating, analyzing, and evaluating buildings, structures, and their layout, form, color, material, and other visual elements within a specific urban or rural area. The purpose is to extract and summarize the unique architectural style, historical context, and landscape characteristics of the area. Identification typically covers multiple dimensions, including the age of the building, functional type, structural form, and decorative details, and is comprehensively considered in conjunction with factors such as the regional environment, cultural background, and historical evolution. Through style identification, we can accurately grasp the characteristics and highlights of urban and rural construction, provide a scientific basis for subsequent style protection, inheritance, and development, and guide urban and rural construction in the direction of continuing the historical context, highlighting regional characteristics, and improving the quality of the human living environment. Style identification is an important foundational work in the fields of urban and rural planning, cultural heritage protection, and tourism development. It is of great significance to maintaining the unique urban and rural style and enhancing cultural confidence.

[0003] However, traditional system updates rely on manually labeled data and are unable to absorb public feedback in real time, resulting in a lag in the understanding of regional architectural characteristics; most methods only perform static classification and lack the ability to predict the evolution trend of style and appearance, making it difficult to promptly discover the protection needs of endangered buildings. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent recognition system for urban and rural architectural features based on multi-source data fusion in order to solve the above-mentioned problems.

[0005] The technical solution adopted by the present invention is as follows: an intelligent recognition system for urban and rural architectural features based on multi-source data fusion, the system comprising: a data acquisition module, a data preprocessing module, a feature extraction module, an emotion-style feature fusion module, a model training and optimization module, an architectural feature recognition module, and a user interaction module;

[0006] The emotion-style feature fusion module is internally provided with an emotion-style association mapping submodule, a multimodal contrast learning submodule and a dynamic feedback optimization submodule;

[0007] The satellite image output terminal of the data acquisition module is connected to the remote sensing image input terminal of the data preprocessing module.

[0008] The multimodal fusion engine of the data preprocessing module is connected to the visual feature branch of the feature extraction module through a spatiotemporal encoder.

[0009] The visual semantic vector output of the feature extraction module is directly connected to the visual input of the cross-modal graph attention network of the emotion-style feature fusion module;

[0010] The emotion-style feature fusion module is deployed to the real-time inference engine of the architectural style recognition module through an encrypted channel. At the same time, the user feedback data flows through the edge computing node after being filtered.

[0011] The model training optimization module is fed back to the dynamic memory bank of the model training optimization module through the incremental learning channel, forming a closed-loop optimization path;

[0012] The multi-terminal application framework of the user interaction module maintains data synchronization with the knowledge graph retrieval engine of the architectural style recognition module through the RESTful API, ensuring that the style distribution heat map retrieved by the visualization system corresponds to the spatiotemporal analysis results of the recognition module in real time.

[0013] In a preferred embodiment, the data acquisition module is internally provided with five types of heterogeneous data acquisition channels; the satellite and aerial image channel uses high-resolution series satellites and unmanned aerial vehicle (UAV) aerial photography equipment to obtain remote sensing images with a resolution of 0.5 meters, focusing on collecting macro features of building roof forms and community layouts; the street view image channel uses a collection vehicle equipped with a multi-eye panoramic camera for regular inspections, combined with a crowdsourcing model to obtain details of the facades of buildings along the street, and automatically generates 360-degree panoramic images every 200 meters.

[0014] In a preferred embodiment, the data preprocessing module realizes multi-source data standardization through a four-layer processing pipeline; the image processing unit adopts a U-Net network for thin cloud removal and atmospheric correction, and uses an ESRGAN model to perform 4x super-resolution reconstruction on low-resolution images.

[0015] In a preferred embodiment, the feature extraction module is configured to construct a three-dimensional feature representation system; the visual feature branch adopts an improved ResNeXt-101 model, and fine-tunes the parameters of the last layer of convolution kernel based on ImageNet pre-training, which is specifically used to capture fine-grained architectural elements such as gable decorations and window lattice patterns; the text semantic branch deploys a domain-adapted BERT model, which enhances the ability to understand style terminology by injecting the professional dictionary of "History of Chinese Architecture" and outputs a 768-dimensional style semantic vector.

[0016] In a preferred embodiment, the emotion-style association mapping submodule first uses a BERT-based sentiment analysis model to extract sentiment semantic vectors from user comments, and simultaneously extracts visual style feature vectors of architectural images using a pre-trained ResNet-152 network. A bilinear attention layer is then constructed to calculate the interaction matrix between the sentiment and style vectors, and a learnable attention weight allocation network is used to generate a sentiment-style association graph. A directional constraint mechanism is introduced in the specific implementation to force sentiment semantics to dominate the weight allocation direction when calculating attention, for example, prioritizing the strong association between the sentiment word "romantic" and the "curved shape" of architectural decorative elements. Finally, a joint representation vector encoding both subjective perception and objective attributes is generated through weighted fusion of feature channels.

[0017] The mathematical expression of the dynamic generation process of bilinear attention weights is:

[0018]

[0019] where e i The feature vector representing the i-th emotional semantic unit has a dimension of 768; s j W represents the visual feature vector of the j-th architectural style element, with a dimension of 1024; e and W s are the learnable parameter matrices for the emotion domain and style domain, with initial dimensions of 768×256 and 1024×256 respectively; d is the scaling factor constant, which is 256; σ represents the improved GELU activation function, which introduces nonlinear transformation to enhance the expressiveness of the model; this formula effectively solves the problem of cross-modal feature dimensionality differences through dimensional compression and feature alignment of the parameter matrix, while the scaling factor design improves the numerical stability of the attention weight calculation.

[0020] In a preferred embodiment, the multimodal contrastive learning submodule realizes the coordinated optimization of architectural visual features and textual sentiment features by constructing a cross-modal semantic alignment space, and adopts an improved triple contrastive learning framework to enhance the semantic consistency between different modalities. The module first creates cross-modal sample pairs: the anchor sample is the architectural image, the positive sample is the semantically matched text description, and the negative sample is the randomly sampled unrelated text. In the feature encoding stage, the 2048-dimensional architectural style features extracted by the visual encoder and the 768-dimensional sentiment semantic features output by the text encoder are respectively mapped to a 256-dimensional shared space through a modal alignment projection layer. The style consistency constraint term is innovatively introduced to simultaneously optimize the visual style clustering density when calculating the contrast loss, ensuring that buildings of similar styles present continuous distribution characteristics in the embedding space. During the training process, a dynamic temperature coefficient adjustment mechanism is adopted to automatically adjust the loss function sensitivity coefficient according to the inter-modal similarity distribution of the current batch of samples, effectively alleviating the optimization difficulty caused by the difference in cross-modal data distribution.

[0021] The mathematical expression of the contrast loss function for fusion style consistency is:

[0022]

[0023] Where s(v i ,t j ) represents the image v i With text t j The cosine similarity of i is the dynamic temperature coefficient calculated from the variance of image style features, with a scalar dimension; λ is the style consistency loss weight hyperparameter, with a default value of 0.5; μ c represents the cluster center of the c-th architectural style in the feature space, with a dimension of 256; f v is a visual feature projection network with an input dimension of 2048 and an output dimension of 256; Sc is the set of image samples belonging to the c-th style in the current batch.

[0024] In a preferred embodiment, the dynamic feedback optimization submodule collects correction annotations of architectural style recognition results through a user interaction interface. For example, when the system misclassifies "Art Deco style" as "Neoclassicism", the correct label submitted by the user will trigger the feature weight adjustment process; the system maintains a dynamic memory library to store high-frequency feedback data, adopts a sliding window sampling strategy to select the most informative samples for model fine-tuning, and designs a negative sample generator to automatically create confusing style combinations to improve model robustness.

[0025] In a preferred embodiment, the model training optimization module is internally provided with:

[0026] The basic model library includes 12 image classification architectures such as EfficientNetV2 and Swin Transformer, and supports dynamic architecture search to automatically match the best model.

[0027] The distributed training framework implements multi-GPU parallel computing based on Horovod and adopts a progressive thawing strategy to fine-tune network parameters layer by layer.

[0028] The building style recognition module is provided with:

[0029] The architectural style recognition module establishes a five-dimensional intelligent analysis system;

[0030] The style classifier integrates a multi-expert model voting mechanism. When the consistency of the ResNet50 and ViT-B / 16 models in determining a certain architectural style is less than 70%, a cross-modal verification process is automatically triggered to check the text description features.

[0031] The age prediction unit constructs a residual time series network and estimates the construction age within a ±5-year error by analyzing the degradation characteristics of building facade materials and the deformation laws of components.

[0032] The knowledge graph search engine has 7 million built-in architectural entity relationships and supports multi-condition combination queries such as "gable roof + blue brick wall + brick carving decoration";

[0033] The 3D visualization system is equipped with a WebGL engine, which can render the recognition results into real-life AR models with style annotations, and supports the generation of spatiotemporal animations of historical changes.

[0034] In a preferred embodiment, the user interaction module is provided with:

[0035] A two-way intelligent interactive ecosystem; the multi-terminal application framework uses Flutter cross-platform technology to achieve three-terminal collaboration: building scanning and recognition on mobile phones, big data dashboards on PCs, and real-scene annotation on AR glasses.

[0036] The visualization system is designed with a layered display function for heat maps, which supports the generation of regional landscape distribution maps based on the classification dimensions of Ming and Qing dwellings and Soviet-style buildings in the early days of the People's Republic of China.

[0037] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0038] 1. In this invention, by integrating users' emotional feedback on buildings with objective visual features, the system can not only accurately determine the type of architectural style, but also understand the subjective differences in perception caused by different styles. For example, when identifying Hui-style architecture, the system can simultaneously capture the user's nostalgic feelings for its "white walls and black tiles", so that travel recommendations can both showcase the architectural features and match the emotional needs of tourists. The dynamic learning mechanism allows the system to continuously absorb fresh feedback from users from all over the world and automatically correct the original model's cognitive bias towards regional architectural features. For example, it can promptly update the detailed distinction standards between southern Fujian red brick houses and Lingnan wok-ear houses, maintaining its adaptability to the ever-changing urban and rural landscapes.

[0039] 2. The present invention enhances the three-dimensional analysis level of urban and rural architectural value assessment. In the context of cultural heritage protection, the combination of professional surveying and mapping data and public sentiment evaluation can more accurately locate modern and contemporary buildings with collective memory value. When planning departments use this system, they can not only obtain a heat map of the distribution of architectural styles, but also see the emotional identification of local residents with various types of buildings, providing a dual basis for urban renewal that takes into account professional standards and public opinion. Real-time updated multi-source data enables the system to promptly detect abnormal changes in endangered buildings in traditional villages, predict the risk of loss of their cultural value through sentimental feature analysis, and strive for a longer response time window for protection decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a block diagram of the overall system of the present invention;

[0041] Figure 2 This is a system block diagram of the emotion-style feature fusion module in the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] Example:

[0044] Reference Figure 1-2 , an intelligent recognition system for urban and rural architectural features based on multi-source data fusion, the system includes: data acquisition module, data preprocessing module, feature extraction module, emotion-style feature fusion module, model training and optimization module, architectural feature recognition module and user interaction module;

[0045] The emotion-style feature fusion module is internally configured with an emotion-style correlation mapping submodule, a multimodal contrast learning submodule, and a dynamic feedback optimization submodule;

[0046] The satellite image output terminal of the data acquisition module is connected to the remote sensing image input terminal of the data preprocessing module.

[0047] The multimodal fusion engine of the data preprocessing module is connected to the visual feature branch of the feature extraction module through the spatiotemporal encoder.

[0048] The visual semantic vector output of the feature extraction module is directly connected to the visual input of the cross-modal graph attention network of the emotion-style feature fusion module.

[0049] The emotion-style feature fusion module is deployed to the real-time inference engine of the architectural style recognition module through an encrypted channel. At the same time, the user feedback data flows through the edge computing node after being filtered.

[0050] The model training optimization module transmits the information back to the dynamic memory bank of the model training optimization module through the incremental learning channel, forming a closed-loop optimization path.

[0051] The multi-terminal application framework of the user interaction module maintains data synchronization with the knowledge graph retrieval engine of the architectural style recognition module through the RESTful API, ensuring that the style distribution heat map retrieved by the visualization system corresponds to the spatiotemporal analysis results of the recognition module in real time.

[0052] There are five types of heterogeneous data acquisition channels set inside the data acquisition module. The satellite and aerial image channel uses high-resolution series satellites and drone aerial photography equipment to obtain remote sensing images with a resolution of 0.5 meters, focusing on collecting macroscopic features such as the form of building roofs and the layout of communities. The street view image channel regularly inspects through a collection vehicle equipped with a multi-camera panoramic camera, and combines the crowdsourcing mode to obtain the details of the building facades along the street. A 360-degree panoramic image is automatically generated every 200 meters. The historical building database is connected to the national list of cultural relics protection units and the results of historical building censuses in various provinces and cities, integrating structured data fields such as the building age, design drawings, and repair records. The social media crawling engine deploys a distributed crawler cluster based on the Scrapy framework to real-time crawl the graphic and text content with tags such as #ancient buildings# and #urban memory# on mainstream platforms, and establish a UGC database with spatio-temporal indexing. The user upload interface supports the upload of JPEG / PNG format images and MP4 short videos, and an EXIF information extraction module is built-in to automatically record the shooting location and time.

[0053] The data preprocessing module realizes the standardization of multi-source data through a four-layer processing pipeline. The image processing unit uses the U-Net network for thin cloud removal and atmospheric correction, and applies the ESRGAN model to perform 4-fold super-resolution reconstruction on low-resolution images. The text cleaning pipeline configures a BERT-CRF joint model. After completing conventional word segmentation and stop word filtering, it focuses on identifying professional terms such as "gable roof" and "arcade style" in architecture. The semi-automated annotation platform integrates an active learning mechanism. When the confidence level of the VGG16 pre-trained model in identifying a certain type of architectural style is lower than 85%, it is automatically pushed to the manual annotation queue. The multi-modal fusion engine constructs a spatio-temporal alignment matrix, maps and associates the GPS coordinates of street view images with the tile numbers of satellite images, and realizes the spatial matching block of the remote sensing top view and street view front view data of a single building.

[0054] The feature extraction module is set up to construct a three-dimensional feature representation system. The visual feature branch uses an improved ResNeXt-101 model, and fine-tunes the parameters of the last convolutional kernel based on the ImageNet pre-training, which is specifically used to capture fine-grained architectural elements such as gable decorations and window lattice patterns. The text semantic branch deploys a domain-adapted BERT model, and enhances the understanding ability of style terms by injecting a professional dictionary of "History of Chinese Architecture", and outputs a 768-dimensional style semantic vector. The emotion-style fusion sub-module innovatively designs a cross-modal graph attention network, converts the emotion polarity value in social media text into a 32-dimensional emotion vector, and performs channel-level attention interaction with the visual feature map to generate a 1024-dimensional joint feature vector that integrates the user's subjective perception.

[0055] The emotion-style association mapping submodule constructs a cross-modal association model by fusing user emotion features with architectural style features. Its core approach utilizes a dual-channel cross-modal attention mechanism to achieve dynamic feature interaction. This module first uses a BERT-based sentiment analysis model to extract sentiment semantic vectors from user reviews. Simultaneously, a pre-trained ResNet-152 network extracts visual style feature vectors from architectural images. A bilinear attention layer is then constructed to calculate the interaction matrix between the sentiment and style vectors. A learnable attention weight allocation network is then used to generate a sentiment-style association graph. A directional constraint mechanism is introduced in the implementation to enforce that sentiment semantics dominate the weight allocation direction during attention calculation. For example, this prioritizes a strong association between the sentiment word "romantic" and the architectural decorative element "curved shape." Finally, a weighted fusion of feature channels is used to generate a joint representation vector that encodes both subjective perceptions and objective attributes.

[0056] The mathematical expression of the dynamic generation process of bilinear attention weights is:

[0057]

[0058] where e i The feature vector representing the i-th emotional semantic unit has a dimension of 768; s j W represents the visual feature vector of the j-th architectural style element, with a dimension of 1024. e and W s d is the learnable parameter matrix for the emotion domain and style domain, with initial dimensions of 768×256 and 1024×256, respectively. d is a scaling factor constant, set to 256. σ represents the improved GELU activation function, which introduces nonlinear transformations to enhance the model's expressiveness. This formula effectively addresses the issue of cross-modal feature dimensionality discrepancies by dimensionality compression and feature alignment of the parameter matrix. The scaling factor design also improves the numerical stability of attention weight calculation.

[0059] The multimodal contrastive learning submodule achieves collaborative optimization of architectural visual features and textual sentiment features by constructing a cross-modal semantic alignment space. It employs an improved triplet contrastive learning framework to enhance semantic consistency across modalities. This module first creates cross-modal sample pairs: anchor samples are architectural images, positive samples are semantically matched text descriptions, and negative samples are randomly sampled unrelated text. During the feature encoding phase, the 2048-dimensional architectural style features extracted by the visual encoder and the 768-dimensional sentiment semantic features output by the text encoder are mapped to a 256-dimensional shared space via a modality alignment projection layer. An innovative style consistency constraint is introduced to simultaneously optimize the closeness of visual style clustering when calculating the contrastive loss, ensuring that buildings of similar styles exhibit a continuous distribution in the embedding space. During training, a dynamic temperature coefficient adjustment mechanism is employed to automatically adjust the loss function sensitivity coefficient based on the inter-modal similarity distribution of the current batch of samples, effectively alleviating optimization difficulties caused by differences in cross-modal data distribution.

[0060] The mathematical expression of the contrast loss function for fusion style consistency is:

[0061]

[0062] Where s(v i ,t j ) represents the image v i With text t j The cosine similarity of i is the dynamic temperature coefficient calculated from the variance of image style features, with a scalar dimension; λ is the style consistency loss weight hyperparameter, with a default value of 0.5; μ c represents the cluster center of the c-th architectural style in the feature space, with a dimension of 256; f v is a visual feature projection network with a 2048-dimensional input and a 256-dimensional output. Sc is the set of image samples belonging to style category c in the current batch. This formula adapts to the differences in feature distribution across different style categories through a dynamic temperature coefficient, while forcing samples of the same style to maintain a compact distribution in the projected space, significantly improving the accuracy of cross-modal retrieval.

[0063] The dynamic feedback optimization submodule collects corrections to architectural style recognition results through a user interface. For example, if the system misclassifies "Art Deco" as "Neoclassical," the correct label submitted by the user triggers a feature weight adjustment process. The system maintains a dynamic memory bank to store high-frequency feedback data and employs a sliding window sampling strategy to select the most informative samples for model fine-tuning. A negative sample generator is also designed to automatically create confusing style combinations to enhance model robustness. For unstructured sentiment expressions actively uploaded by users, such as "This church's spire makes me feel sacred," the module integrates a semantic parsing network to extract implicit stylistic cues. These cues are then mapped to an existing style classification system through feature space projection, dynamically expanding the emotional relevance of architectural styles. During training, elastic weight solidification is employed to protect important features of historical data from being overwritten when updating model parameters, ensuring the stability of the system's existing knowledge as it adapts to new feedback.

[0064] The internal settings of the model training optimization module are:

[0065] The basic model library includes 12 image classification architectures, including EfficientNetV2 and Swin Transformer, and supports dynamic architecture search to automatically match the optimal model. The distributed training framework uses Horovod to implement multi-GPU parallel computing and employs a progressive thawing strategy to fine-tune network parameters layer by layer. In addition to conventional accuracy metrics, the model evaluation system incorporates a unique style confusion matrix to quantitatively analyze the probability of misclassification for easily confused styles such as New Chinese and Japanese dry landscape architecture. The edge deployment solution uses the TensorRT engine to perform FP16 quantization and compression on the model, increasing inference speed to real-time processing at 15 frames per second while maintaining 98% accuracy.

[0066] The building style recognition module is set up with:

[0067] The architectural style recognition module establishes a five-dimensional intelligent analysis system. The style classifier integrates a multi-expert model voting mechanism. When the consensus of the three models, ResNet50, ViT-B / 16, on a particular architectural style falls below 70%, a cross-modal verification process is automatically triggered to verify the textual description features. The age prediction unit constructs a residual time series network, analyzing the degradation characteristics of building facade materials and the deformation patterns of components to estimate the construction age within a ±5-year error. The knowledge graph search engine contains 7 million built-in architectural entity relationships and supports multi-condition combination queries such as "gable roof + blue brick wall + brick carving decoration." The 3D visualization system, equipped with a WebGL engine, can render recognition results into real-life AR models with style annotations, supporting the generation of spatiotemporal animations of historical style changes.

[0068] The user interaction module is set up with:

[0069] A two-way intelligent interactive ecosystem. The multi-terminal application framework utilizes Flutter's cross-platform technology to achieve three-way collaboration: mobile phone architectural scanning and recognition, PC-based big data dashboards, and AR glasses-based real-scene annotation. The visualization system features a layered heat map display function, supporting the generation of regional landscape distribution maps based on categories such as Ming and Qing dynasty dwellings and early-stage Soviet-style architecture. The feedback optimization mechanism features a dual-channel entry point. Regular users can submit their satisfaction ratings using a five-star rating system, while professional users can directly modify style classification labels in the error annotation correction interface. The privacy protection layer implements end-to-end data encryption. After the edge computing device completes feature extraction on user-uploaded images, the original data is immediately deleted, retaining only the desensitized feature vectors for model optimization.

[0070] From the above we can know:

[0071] In this invention, by integrating users' emotional feedback on buildings with objective visual features, the system can not only accurately determine architectural style types but also understand the subjective differences in perceptions caused by different styles. For example, when identifying Hui-style architecture, the system can simultaneously capture the user's nostalgic feelings for its "white walls and black tiles," allowing travel recommendations to both showcase architectural features and meet tourists' emotional needs. A dynamic learning mechanism allows the system to continuously absorb fresh feedback from users across the globe, automatically correcting the original model's cognitive biases regarding regional architectural characteristics. For example, it can promptly update the detailed distinction criteria between southern Fujian red brick houses and Lingnan wok-ear houses, maintaining its adaptability to the ever-changing urban and rural landscapes.

[0072] In this invention, the three-dimensional analysis level of urban and rural architectural value assessment is enhanced. In the cultural heritage protection scenario, the combination of professional surveying and mapping data and the dual dimensions of public emotional evaluation can more accurately locate modern and contemporary buildings with collective memory value. When planning departments use this system, they can not only obtain a heat map of the distribution of architectural styles, but also see the emotional identification of local residents with various types of buildings, providing a dual basis for urban renewal that takes into account professional standards and public opinion. Real-time updated multi-source data enables the system to promptly detect abnormal changes in endangered buildings in traditional villages, predict the risk of loss of their cultural value through emotional feature analysis, and strive for a longer response time window for protection decisions.

[0073] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0074] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An intelligent recognition system for urban and rural architectural features based on multi-source data fusion, characterized by: The system includes: a data acquisition module, a data preprocessing module, a feature extraction module, an emotion-style feature fusion module, a model training and optimization module, an architectural style recognition module and a user interaction module; The emotion-style feature fusion module is internally provided with an emotion-style association mapping submodule, a multimodal contrast learning submodule and a dynamic feedback optimization submodule; The satellite image output terminal of the data acquisition module is connected to the remote sensing image input terminal of the data preprocessing module. The multimodal fusion engine of the data preprocessing module is connected to the visual feature branch of the feature extraction module through a spatiotemporal encoder. The visual semantic vector output of the feature extraction module is directly connected to the visual input of the cross-modal graph attention network of the emotion-style feature fusion module; The emotion-style feature fusion module is deployed to the real-time inference engine of the architectural style recognition module through an encrypted channel. At the same time, the user feedback data flows through the edge computing node after being filtered. The model training optimization module is fed back to the dynamic memory bank of the model training optimization module through the incremental learning channel, forming a closed-loop optimization path; The multi-terminal application framework of the user interaction module maintains data synchronization with the knowledge graph retrieval engine of the architectural style recognition module through the RESTful API, ensuring that the style distribution heat map retrieved by the visualization system corresponds to the spatiotemporal analysis results of the recognition module in real time.

2. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized by: The data acquisition module is internally equipped with five types of heterogeneous data acquisition channels; the satellite and aerial image channels use high-resolution satellites and drone aerial photography equipment to obtain remote sensing images with a resolution of 0.5 meters, focusing on collecting macro features of building roof forms and community layouts; The street view image channel is regularly inspected by a collection vehicle equipped with multiple panoramic cameras. Combined with a crowdsourcing model, it obtains details of the facades of buildings along the street and automatically generates 360-degree panoramic images every 200 meters.

3. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized in that: The data preprocessing module achieves multi-source data standardization through a four-layer processing pipeline; the image processing unit adopts the U-Net network to remove thin clouds and perform atmospheric correction, and uses the ESRGAN model to perform 4x super-resolution reconstruction on low-resolution images.

4. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized in that: The feature extraction module is provided with a system for constructing three-dimensional feature representation; The visual feature branch uses an improved ResNeXt-101 model. Based on ImageNet pre-training, the parameters of the last layer of convolutional kernels are fine-tuned to capture fine-grained architectural elements such as gable decorations and window mullion patterns. The text semantics branch deploys a domain-adapted BERT model, which enhances the ability to understand style terminology by injecting the professional dictionary of "History of Chinese Architecture" and outputs a 768-dimensional style semantic vector.

5. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized in that: The emotion-style association mapping submodule first uses a BERT-based sentiment analysis model to extract sentiment semantic vectors from user reviews, and simultaneously extracts visual style feature vectors from architectural images using a pre-trained ResNet-152 network. A bilinear attention layer is then constructed to calculate the interaction matrix between the sentiment and style vectors, and a learnable attention weight allocation network is used to generate a sentiment-style association graph. A directional constraint mechanism is introduced in the implementation to force sentiment semantics to dominate the weight allocation direction when calculating attention. For example, a strong association is prioritized between the sentiment word "romantic" and the architectural decorative element "curved shape." Finally, a weighted fusion of feature channels is used to generate a joint representation vector that encodes both subjective perception and objective attributes. The mathematical expression of the dynamic generation process of bilinear attention weights is: where e i The feature vector representing the i-th emotional semantic unit has a dimension of 768; s j W represents the visual feature vector of the j-th architectural style element, with a dimension of 1024; e and W s are the learnable parameter matrices for the emotion domain and style domain, with initial dimensions of 768×256 and 1024×256 respectively; d is the scaling factor constant, which is 256; σ represents the improved GELU activation function, which introduces nonlinear transformation to enhance the expressiveness of the model; this formula effectively solves the problem of cross-modal feature dimensionality differences through dimensional compression and feature alignment of the parameter matrix, while the scaling factor design improves the numerical stability of the attention weight calculation.

6. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized in that: The multimodal contrastive learning submodule achieves the coordinated optimization of architectural visual features and textual sentiment features by constructing a cross-modal semantic alignment space, and adopts an improved triple contrastive learning framework to enhance the semantic consistency between different modalities. This module first creates cross-modal sample pairs: the anchor sample is the architectural image, the positive sample is the semantically matched text description, and the negative sample is randomly sampled unrelated text. During the feature encoding stage, the 2048-dimensional architectural style features extracted by the visual encoder and the 768-dimensional sentiment semantic features output by the text encoder are mapped to a 256-dimensional shared space through a modal alignment projection layer. A style consistency constraint is innovatively introduced to simultaneously optimize the compactness of visual style clustering when calculating the contrast loss, ensuring that buildings of similar styles exhibit a continuous distribution in the embedding space. During training, a dynamic temperature coefficient adjustment mechanism is employed to automatically adjust the loss function sensitivity coefficient based on the inter-modal similarity distribution of the current batch of samples, effectively alleviating optimization difficulties caused by differences in cross-modal data distribution. The mathematical expression of the contrast loss function for fusion style consistency is: Where s(v i ,t j ) represents the image v i With text t j The cosine similarity of i is the dynamic temperature coefficient calculated from the variance of image style features, with a scalar dimension; λ is the style consistency loss weight hyperparameter, with a default value of 0.5; μ c represents the cluster center of the c-th architectural style in the feature space, with a dimension of 256; f v is a visual feature projection network with an input dimension of 2048 and an output dimension of 256; Sc is the set of image samples belonging to the c-th style in the current batch.

7. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized in that: The dynamic feedback optimization submodule collects correction annotations of architectural style recognition results through a user interaction interface. For example, when the system misclassifies "Art Deco" as "Neoclassicism," the correct label submitted by the user triggers a feature weight adjustment process. The system maintains a dynamic memory library to store high-frequency feedback data, uses a sliding window sampling strategy to select the most informative samples for model fine-tuning, and designs a negative sample generator to automatically create confusing style combinations to improve model robustness.

8. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized in that: The internal settings of the model training optimization module are: The basic model library includes 12 image classification architectures such as EfficientNetV2 and Swin Transformer, and supports dynamic architecture search to automatically match the best model. The distributed training framework implements multi-GPU parallel computing based on Horovod and adopts a progressive thawing strategy to fine-tune network parameters layer by layer.

9. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1 is characterized in that: The building style recognition module is provided with: The architectural style recognition module establishes a five-dimensional intelligent analysis system; The style classifier integrates a multi-expert model voting mechanism. When the consistency of the ResNet50 and ViT-B / 16 models in determining a certain architectural style is less than 70%, a cross-modal verification process is automatically triggered to check the text description features. The age prediction unit constructs a residual time series network and estimates the construction age within a ±5-year error by analyzing the degradation characteristics of building facade materials and the deformation laws of components. The knowledge graph search engine has 7 million built-in architectural entity relationships and supports multi-condition combination queries such as "gable roof + blue brick wall + brick carving decoration"; The 3D visualization system is equipped with a WebGL engine, which can render the recognition results into real-life AR models with style annotations, and supports the generation of spatiotemporal animations of historical changes.

10. The intelligent recognition system for urban and rural architectural features based on multi-source data fusion according to claim 1, characterized in that: The user interaction module is provided with: A two-way intelligent interactive ecosystem; the multi-terminal application framework uses Flutter cross-platform technology to achieve three-terminal collaboration: building scanning and recognition on mobile phones, big data dashboards on PCs, and real-scene annotation on AR glasses. The visualization system is designed with a layered display function for heat maps, which supports the generation of regional landscape distribution maps based on building classification dimensions.

Images