Shikumen building style value evaluation method based on large model
Patent Information
- Application Number
- CN202611141595.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-30
- Publication Date
- 2026-10-30
AI Technical Summary
[0006]本发明的目的在于提供一种基于大模型的风貌建筑石库门价值评估方法，能够解决现有技术中评估粒度粗、可解释性差、小样本场景泛化能力不足的技术问题
[0038]1、本发明能够大幅提升评估效率，支撑规模化应用，实现了石库门风貌价值的自动化评估，单张图像处理耗时可达秒级，相比人工现场勘查评估效率提升数百倍，可支撑城市级石库门建筑的批量普查，显著降低人力成本与时间成本。
Smart Images
Figure CN123016066A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the interdisciplinary field of computer vision and historical building preservation, and in particular to a method for assessing the value of Shikumen (stone gate) buildings based on large models. Background Technology
[0002] Shikumen architecture is a highly representative historical architectural style in Shanghai and an important part of the city's historical and cultural heritage. The scientific assessment of its stylistic value is a core foundation for decisions regarding the restoration of historical buildings, the classification of protection areas, and daily operation and maintenance. Currently, the assessment of the stylistic value of Shikumen buildings mainly relies on on-site inspections by experts, who use their professional experience to judge the integrity of architectural components, decorative features, and the degree of preservation of the stylistic features, and then comprehensively assign a value grade.
[0003] As the protection of historical buildings in cities is carried out on a large scale, traditional manual assessment methods have gradually revealed many shortcomings: First, the assessment efficiency is low. On-site inspection and assessment of a single building requires a lot of manpower and time, making it difficult to support a large-scale survey of Shikumen buildings at the city level. Second, the assessment is highly subjective. The assessment results depend heavily on the personal experience of experts, and the judgment standards of different personnel are different, resulting in insufficient consistency and reproducibility of the results. Third, the cost is high. Large-scale surveys require a large number of professional personnel, resulting in high human resource costs.
[0004] At present, artificial intelligence image recognition technology has begun to be gradually applied to the field of building inspection, but existing solutions have obvious limitations: one type of solution uses an end-to-end image classification model to directly output the value level, which is a black box judgment that cannot clearly identify specific components or provide evaluation basis. The results lack interpretability and are difficult to support protection decisions. Another type of solution uses traditional target detection algorithms to identify building components, but it can only complete the location of components and cannot complete the graded value assessment by combining the rules of form combination. Moreover, for the scenario of diverse Shikumen forms and scarce labeled samples, the model has poor generalization ability and the recognition accuracy is difficult to meet the engineering requirements.
[0005] If value levels are directly generated from single images using large models, problems such as unclear level criteria, instability from multiple outputs of the same image, and difficulty in batch parsing of results can easily arise. Therefore, there is a need to provide a value assessment method for Shikumen (stone-gate) buildings based on large models, which can solve the technical problems of coarse assessment granularity, poor interpretability, and insufficient generalization ability for small sample scenarios in existing technologies. Summary of the Invention
[0006] The purpose of this invention is to provide a method for evaluating the value of Shikumen (stone gate) buildings based on a large model, which can solve the technical problems of coarse evaluation granularity, poor interpretability, and insufficient generalization ability of small sample scenarios in the existing technology.
[0007] To achieve the above objectives, the technical solution of the present invention is as follows:
[0008] A method for assessing the value of Shikumen (stone-gate) architecture based on a large model includes the following steps:
[0009] Step 1: Assess the necessary preparations;
[0010] Step 2: Input a single frontal image of a Shikumen building into a multimodal large model;
[0011] Step 3: Task distribution;
[0012] Step 4: Load the dedicated LoRA adapter for Shikumen component identification;
[0013] Step 5: Output the list of deterministic components;
[0014] Step 6: Structured parsing;
[0015] Step 7: Standardize value grading;
[0016] Step 8: Evaluation results generation, output, and archiving.
[0017] Step 1 includes the following sub-steps:
[0018] Step 11: Setting up a multimodal large model environment;
[0019] Step 12: Fine-tune dataset construction and model adaptation;
[0020] Step 13: Evaluate the rule base configuration.
[0021] In step 11, a multimodal large model is deployed on a local GPU server as a vision-language joint inference model, and an inference environment is built based on the Torch framework, with a unified HTTP inference interface encapsulated through FastAPI.
[0022] In step 12, a data augmentation strategy based on fine-grained semantic deconstruction is used to construct training data: collected annotated images of Shikumen buildings, decomposed a single image containing N component annotations into N independent atomic binary classification question-answering samples, balanced the data distribution through global shuffling and positive and negative sample resampling, and expanded the scale of effective training data; based on LoRA technology, the parameters of the multimodal large model are efficiently fine-tuned, the main parameters of the base model are frozen, and a low-rank adapter is injected only into the projection matrix of the attention layer to train a dedicated LoRA adapter for Shikumen component recognition.
[0023] In step 13, a list of core components of a Shikumen, a classification standard for component shapes, a mapping rule for component-style value, and a grading rule for style value are preset and stored in the backend in the form of a structured configuration file; the core components include the door frame, lintel, door leaf, and special decorations.
[0024] In step 3, after receiving the front image of the Shikumen building, the landscape value assessment task is automatically triggered, and the task type is specified as component recognition.
[0025] In step 5, the LoRA adapter for identifying Shikumen components uses a deterministic generation strategy to call a multimodal large model to perform inference: the random sampling mechanism is turned off, and the output format of the multimodal large model is constrained by preset standardized structured constraint prompts.
[0026] In step 6, a regular expression symbol extraction algorithm is used to scan the list of deterministic components output by the multimodal large model line by line, extract the key-value pairs of the component names and judgment results of the core components in the list of deterministic components, and convert the unstructured natural language text into a structured JSON data object.
[0027] In step 6, the original output text of the multimodal large model is retained simultaneously as a source for tracing the evaluation results.
[0028] In step 7, based on the hit data of core components in the deterministic component list, the feature value classification is determined by the rule engine. The determination logic is as follows:
[0029] The rule engine first determines the corresponding style value level of the core components based on the component-style value mapping rule and the style value grading rule, and then uses the highest style value level as the overall style value level of the Shikumen building.
[0030] The core components include: door frame, lintel, and door leaf;
[0031] When two or more core components are detected to have high-value forms, the building is judged to have a complete form and a high degree of preservation of historical features, corresponding to a high value level.
[0032] When a core component is detected to have a high-value form, it corresponds to a medium-value level;
[0033] When no core components with high-value form are detected, or when the list of definitive components shows that typical Shikumen features are missing, the building is judged to have undergone serious alteration or lost typical Shikumen features, corresponding to a low value level.
[0034] High-value features include: stone slabs, brick carvings and blue tile roofing, triangular floral decorations, semi-circular floral decorations, arched floral decorations, solid wood doors with copper door knockers, inscriptions by literati or auspicious four-character horizontal scrolls, lane names and construction years engraved on the door lintel, and Western classical pilasters on both sides of the door frame.
[0035] When the image visibility is insufficient, the system outputs a verification mark.
[0036] In step 8, the overall style value level, the hit list of structural components, and the basis for style value grading are integrated to generate a standardized assessment report, which is then fed back to the user through the front-end interface. At the same time, it supports batch archiving of assessment results and access to the historical building protection and management database.
[0037] Compared with the prior art, the present invention has the following beneficial effects:
[0038] 1. This invention can significantly improve assessment efficiency, support large-scale application, realize automated assessment of the value of Shikumen architectural style, and the processing time of a single image can be as low as seconds. Compared with manual on-site survey and assessment, the efficiency is improved by hundreds of times. It can support the batch survey of Shikumen buildings at the city level and significantly reduce labor and time costs.
[0039] 2. This invention can ensure uniform evaluation standards and objective and reproducible results. It adopts a fully deterministic process of "multimodal large model recognition + landscape value grading rule judgment", which eliminates subjective bias caused by differences in human experience. The evaluation execution standards of different batches and different regions are completely uniform, and the evaluation results have a high degree of consistency and reproducibility.
[0040] 3. The evaluation process of this invention is traceable and the evaluation results are interpretable. Compared with the black-box end-to-end classification model of the prior art, this invention outputs a clear list of core components and a classification judgment basis. The evaluation process is traceable and the results are interpretable, which can directly support management decisions such as the protection classification and repair priority ranking of historical buildings.
[0041] 4. This invention can be adapted to small sample scenarios and has low application costs. Through data augmentation with fine-grained semantic deconstruction and efficient fine-tuning of LoRA parameters, high-precision component recognition can be achieved with a small number of labeled samples. It is suitable for the diverse forms of Shikumen and the scarcity of labeled data, thus reducing the computing power and data costs of application.
[0042] 5. This invention evaluates each component separately through the component-feature value mapping rule, and forms an overall conclusion based on the highest feature value. This can transform the feature value of Shikumen from a single image level judgment to a component-level, rule-based, and verifiable evaluation process. Attached Figure Description
[0043] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.
[0044] Figure 1 This is a flowchart of the method for evaluating the value of Shikumen (stone gate) buildings based on a large model, as proposed in this invention.
[0045] Figure 2 This is a flowchart of the valuation process for Shikumen (stone gate) buildings based on a large model, as described in this invention. Detailed Implementation
[0046] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more comprehensive overview of the proposed method for assessing the value of Shikumen (stone-gate) architecture based on a large model. The advantages and features of this invention will become clearer from the following description and claims. It should be noted that the accompanying drawings are in a very simplified form and use non-precise scales, intended only to facilitate and clarify the illustration of the embodiments of this invention.
[0047] Please see the appendix Figure 1 and attached Figure 2 A method for assessing the value of Shikumen (stone-gate) architecture based on a large model includes the following steps:
[0048] Step 1: Assess the necessary preparations.
[0049] Step 1 includes the following sub-steps:
[0050] Step 11: Building a multimodal large model environment.
[0051] Specifically, a multimodal large model (preferably Qwen3-VL series) is deployed on a local GPU server as a vision-language joint inference model, and an inference environment is built based on the Torch framework. A unified HTTP inference interface is encapsulated through FastAPI to support local network calls and public network mapping access.
[0052] Step 12: Fine-tune dataset construction and model adaptation.
[0053] Specifically, a data augmentation strategy based on fine-grained semantic deconstruction is employed to construct training data: Annotated images of Shikumen buildings are collected, and a single image containing N component annotations is decomposed into N independent atomic binary classification question-answering samples (e.g., "Does the annotated image of a Shikumen building contain a Shikumen door frame?"). Global shuffling and positive / negative sample resampling are used to balance the data distribution and expand the effective training data scale. Based on LoRA (Low-Rank Adaptation) technology, the parameters of the multimodal large model are efficiently fine-tuned. The main parameters of the base model are frozen, and a low-rank adapter is injected only into the attention layer projection matrix, thus training a dedicated LoRA adapter for Shikumen component recognition.
[0054] Step 13: Evaluate the rule base configuration.
[0055] Specifically, a pre-defined list of core components of Shikumen (including typical core components such as door frames, lintels, door panels, and special decorations), component form classification standards, component-style value mapping rules, and style value grading rules are stored in the backend in the form of a structured configuration file, which supports flexible adjustments.
[0056] The types of core components, the classification standards for component shapes, the mapping rules for component-style value, and the grading rules for style value can be adaptively specified according to actual usage needs, and will not be elaborated here.
[0057] Step 2: Input a single front view image (architectural photo) of a Shikumen building into a multimodal large model.
[0058] Preferably, users can upload images of the front of a Shikumen building via a front-end mini-program. Strict shooting conditions are not required for the input image; only the core components of the Shikumen, such as the doorway, door frame, and lintel, need to be fully visible.
[0059] Step 3: Task distribution.
[0060] Specifically, after receiving the front image of a Shikumen building, the system automatically triggers a landscape value assessment task, specifying the task type as component recognition.
[0061] Step 4: Load the dedicated LoRA adapter for Shikumen component identification.
[0062] Step 5: Output the list of deterministic components.
[0063] In step 5, the LoRA adapter for identifying Shikumen components uses a deterministic generation strategy to call a multimodal large model to perform inference: the random sampling mechanism is turned off (do_sample=False), and the output format of the multimodal large model is constrained by preset standardized structured constraint prompts.
[0064] Preferably, the standardized structured constraint prompts contain multiple layers of mandatory constraint instructions to precisely regulate the model's output logic: First, output format constraint instructions force the model to output only structured list text, prohibiting the output of explanatory text, transitional statements, subjective evaluations, redundant remarks, and other irrelevant content. All output content must strictly correspond to the list fields. Second, field order constraint instructions completely replicate the fixed order of all core components such as door frames, lintels, door panels, and special decorations in the "Shikumen Component Form Survey List," forcing the model to strictly output results according to the preset field order, without changing the field order, omitting any, or adding any irrelevant fields. Third, layered output instructions for field content set uniform output rules for each component field: prioritize outputting a binary existence judgment result of "yes / no." If the judgment result is "yes," then fill in the specific form type corresponding to the component immediately after the judgment result, such as the triangular, semi-circular, or arc-shaped decorations of the Shikumen lintel, achieving standardized and normalized output of each field's content. Step 6: Structured Parsing.
[0065] Specifically, a regular expression symbol extraction algorithm is used to scan the list of deterministic components output by the multimodal large model line by line, extract the key-value pairs of the component names and judgment results of the core components in the list of deterministic components, and convert the unstructured natural language text into structured JSON data objects.
[0066] In step 6, the original output text of the multimodal large model is retained simultaneously as a source for tracing the evaluation results.
[0067] Step 7: Standardize value grading.
[0068] Specifically, based on the hit data of core components in the deterministic component list, the rule engine performs a landscape value classification determination, and the determination logic is as follows:
[0069] The rule engine first determines the corresponding style value level for core components such as door frames, lintels, door panels, and special decorations based on the component-style value mapping rule and the style value grading rule. Then, the highest style value level is used as the overall style value level of the Shikumen building. When two or more core components are detected to have high-value forms, the building is judged to have a complete form and a high degree of preservation of historical features, corresponding to a high value level.
[0070] When a core component is detected to have a high-value form, it corresponds to a medium-value level. Core components include: door frame, lintel, door leaf, and special decorations.
[0071] When no core components with high-value forms are detected, the building is judged to have undergone serious alterations or lost its typical Shikumen features, corresponding to a low value level.
[0072] Higher-value designs include: stone slabs, brick carvings with blue tile roofs, triangular floral decorations, semi-circular floral decorations, arched floral decorations, solid wood doors with copper door knockers, inscriptions by literati or auspicious four-character horizontal scrolls, door lintels with the name of the alley and the year of construction, and Western classical pilasters on both sides of the door frame.
[0073] When image visibility is insufficient, the system outputs a verification mark to prevent low-confidence results from directly entering the final classification.
[0074] Among them, the criteria for judging insufficient image visibility include: large-area occlusion in the image (vegetation or debris occlusion accounting for more than 50%), and the image being too dark / too bright, making it impossible to identify component features.
[0075] Step 8: Generate, output, and archive the evaluation results (construction list and judgment criteria).
[0076] Specifically, it integrates three parts: the overall style value level, the hit list of structural components, and the basis for style value grading, to generate a standardized assessment report, which is then fed back to the user through the front-end interface; it also supports batch archiving of assessment results and access to the historical building protection and management database.
[0077] This invention uses a multimodal large model as the core of visual perception. Through data augmentation strategies based on fine-grained semantic deconstruction and efficient fine-tuning of LoRA parameters, it achieves accurate identification of the core components of Shikumen under small sample conditions. Then, through a structured rule engine, it completes the classification of landscape value based on the number of core component hits, the combination relationship of core components, and the component-landscape value mapping rules. Finally, it outputs an interpretable evaluation result that includes the overall landscape value level, the basis for the landscape value classification, and the hit list of structured components.
[0078] This invention separates model perception from rule-based decision-making, preventing the multimodal large model from directly determining the final value level. It breaks down the evaluation of Shikumen architectural style into two stages: component-level fact identification and rule-based grading. By standardizing component lists, deterministic generation, and structured analysis, it reduces the uncertainty of the large model's output, achieving end-to-end automated evaluation. This provides efficient, unified, and traceable technical support for large-scale surveys, protection grading, and restoration decisions of historical buildings.
[0079] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.
Claims
1. A method for evaluating the value of Shikumen (stone-gate) architecture based on a large model, characterized in that, Includes the following steps: Step 1: Assess the necessary preparations; Step 2: Input a single frontal image of a Shikumen building into a multimodal large model; Step 3: Task distribution; Step 4: Load the dedicated LoRA adapter for Shikumen component identification; Step 5: Output the list of deterministic components; Step 6: Structured parsing; Step 7: Standardize value grading; Step 8: Evaluation results generation, output, and archiving.
2. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 1, characterized in that, Step 1 includes the following sub-steps: Step 11: Setting up a multimodal large model environment; Step 12: Fine-tune dataset construction and model adaptation; Step 13: Evaluate the rule base configuration.
3. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 2, characterized in that, In step 11, a multimodal large model is deployed on a local GPU server as a vision-language joint inference model, and an inference environment is built based on the Torch framework, with a unified HTTP inference interface encapsulated through FastAPI.
4. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 2, characterized in that, In step 12, a data augmentation strategy based on fine-grained semantic deconstruction is used to construct training data: collected annotated images of Shikumen buildings, decomposed a single image containing N component annotations into N independent atomic binary classification question-answering samples, balanced the data distribution through global shuffling and positive and negative sample resampling, and expanded the scale of effective training data; based on LoRA technology, the parameters of the multimodal large model are efficiently fine-tuned, the main parameters of the base model are frozen, and a low-rank adapter is injected only into the projection matrix of the attention layer to train a dedicated LoRA adapter for Shikumen component recognition.
5. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 2, characterized in that, In step 13, a list of core components of a Shikumen, a classification standard for component shapes, a mapping rule for component-style value, and a grading rule for style value are preset and stored in the backend in the form of a structured configuration file; the core components include the door frame, lintel, door leaf, and special decorations.
6. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 1, characterized in that, In step 3, after receiving the front image of the Shikumen building, the landscape value assessment task is automatically triggered, and the task type is specified as component recognition.
7. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 1, characterized in that, In step 5, the LoRA adapter for identifying Shikumen components uses a deterministic generation strategy to call a multimodal large model to perform inference: the random sampling mechanism is turned off, and the output format of the multimodal large model is constrained by preset standardized structured constraint prompts.
8. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 1, characterized in that, In step 6, a regular expression symbol extraction algorithm is used to scan the list of deterministic components output by the multimodal large model line by line, extract the key-value pairs of the component names and judgment results of the core components in the list of deterministic components, and convert the unstructured natural language text into a structured JSON data object. In step 6, the original output text of the multimodal large model is retained simultaneously as a source for tracing the evaluation results.
9. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 1, characterized in that, In step 7, based on the hit data of core components in the deterministic component list, the feature value classification is determined by the rule engine. The determination logic is as follows: The rule engine first determines the corresponding style value level of the core components based on the component-style value mapping rule and the style value grading rule, and then uses the highest style value level as the overall style value level of the Shikumen building. The core components include: door frame, lintel, and door leaf; When two or more core components are detected to have high-value forms, the building is judged to have a complete form and a high degree of preservation of historical features, corresponding to a high value level. When a core component is detected to have a high-value form, it corresponds to a medium-value level; When no core components with high-value form are detected, or when the list of definitive components shows that typical Shikumen features are missing, the building is judged to have undergone serious alteration or lost typical Shikumen features, corresponding to a low value level. High-value features include: stone slabs, brick carvings and blue tile roofing, triangular floral decorations, semi-circular floral decorations, arched floral decorations, solid wood doors with copper door knockers, inscriptions by literati or auspicious four-character horizontal scrolls, lane names and construction years engraved on the door lintel, and Western classical pilasters on both sides of the door frame. When the image visibility is insufficient, the system outputs a verification mark.
10. The method for evaluating the value of Shikumen (stone-gate) architecture based on a large model as described in claim 1, characterized in that, In step 8, the overall style value level, the hit list of structural components, and the basis for style value grading are integrated to generate a standardized assessment report, which is then fed back to the user through the front-end interface. At the same time, it supports batch archiving of assessment results and access to the historical building protection and management database.