Interactive control methods, electronic devices, and computer program products for digital modeling of historical buildings

Abstract

This disclosure provides an interactive control method, electronic device, and computer program product for digital modeling of historical buildings. The interactive control method for digital modeling of historical buildings includes: acquiring point cloud data and images of the historical building to establish a 3D visualization model; identifying components of the historical building from the 3D visualization model using a deep learning algorithm based on the geometric and texture information of the 3D visualization model; matching the components with template components in the HBIM template library; obtaining the component's attribute information from the archive database and binding it to the component based on the matched template component to obtain a digital model of the historical building; structuring and organizing the story clues in the component's attribute information to establish knowledge graph nodes for the component; and responding to the user's first interactive command to click on a target component in the digital model of the historical building, generating personalized explanatory information by calling a large language model based on the user profile and the target component's knowledge graph nodes.

Description

Technical Field

[0001] This disclosure relates to the field of digital modeling of historical buildings, and in particular to an interactive control method for digital modeling of historical buildings, an electronic device, and a computer program product. Background Technology

[0002] In the preservation of historical buildings, digital modeling can accurately record their condition, improving the efficiency and reducing costs of preservation.

[0003] However, the current level of automation in digital modeling of historical buildings is not high, and the information that digital models can present is also limited, which cannot meet the needs of historical building protection work. Efficiency and cost urgently need to be further improved. Summary of the Invention

[0004] This disclosure provides an interactive control method for digital modeling of historical buildings, an electronic device, and a computer program product.

[0005] In a first aspect, embodiments of this disclosure provide an interactive control method for digital modeling of historical buildings, comprising: acquiring point cloud data and images of the historical building; establishing a three-dimensional visualization model of the historical building, the three-dimensional visualization model containing geometric and texture information of the historical building; identifying components of the historical building from the three-dimensional visualization model using a deep learning algorithm based on the geometric and texture information; matching the components with template components in an HBIM template library, wherein the HBIM template library includes various template components and their geometric and texture information; obtaining attribute information of the component from an archive database and binding it to the component based on the matched template component, thereby obtaining a digital model of the historical building; structuring and organizing the story clues in the attribute information of the component to establish knowledge graph nodes of the component, wherein one component is associated with multiple story clues, the story clues including at least one of historical background, architectural style, personal stories, and repair records; and responding to a first interactive instruction from a user to click on a target component in the digital model of the historical building, generating personalized explanatory information by calling a large language model based on the user profile and the knowledge graph nodes of the target component.

[0006] In some embodiments, identifying components of the historical building from the 3D visualization model using a deep learning algorithm based on the geometric information and the texture information includes: performing point cloud segmentation on the 3D visualization model using a deep learning algorithm to extract the geometric shape information of the components of the historical building; classifying the components using a deep learning algorithm to determine the texture feature information of the components; and using a component classification model to perform target detection and style recognition based on the geometric shape information and texture feature information of the components to determine the type and style information of the components.

[0007] In some embodiments, based on the user profile and the knowledge graph nodes of the target component, a large language model is invoked to generate personalized explanation information, including: obtaining user information, identifying the user's interaction intent, and generating a user profile; for structured stories, a preset story template is invoked to generate personalized explanation information based on the user profile; for unstructured stories, a large language model is invoked to generate personalized explanation information based on the user profile.

[0008] In some embodiments, the attribute information of the component includes at least one of material, style, state, and event.

[0009] In some embodiments, the interactive control method for digital modeling of historical buildings further includes: obtaining the lifecycle events corresponding to the historical building from the archive database, parsing them on a component-by-component basis to obtain event information of the components, wherein the event information of the components includes at least one of time, location, participants, event description, and state comparison; establishing an event data model, wherein the event data model represents the event information from time, space, and content dimensions, and associates the events with the components; and visually displaying the events associated with the target component in response to a second interactive instruction from a user clicking on a target component in the digital model of the historical building.

[0010] In some embodiments, the method further includes: acquiring an event image, recognizing the event image, extracting event information from the event image, recognizing components in the event image, updating the extracted event information to the event data model, and associating it with the recognized components.

[0011] In some embodiments, the events associated with the target component are visualized, including: visualizing the events associated with the target component in the form of a timeline; highlighting components that experienced events in the same period in the digital model of the historical building based on the event information of the component; and comparing and displaying the changes of the same component in different periods based on the event information of the component.

[0012] In some embodiments, classifying the components using a deep learning algorithm to determine the texture feature information of the components includes: acquiring sample component images and annotating the sample component images; performing target detection on the sample components in the sample component images based on the YOLOv8 framework, and extracting the bounding boxes and category labels of the sample components; aligning the image features in the bounding boxes of the sample components with the category labels using the CLIP model to perform joint modeling of the image and semantic space to train the classification model; and using the trained classification model to classify the components and determine the texture feature information of the components.

[0013] In some embodiments, the method further includes: parsing the historical archive data of the historical building, extracting feature templates, and establishing a component feature library, wherein the feature templates represent location information, layout information, component information, etc., and the component feature library includes multiple two-dimensional vector contours and three-dimensional point cloud slices with labels of age, style, material, and craftsmanship; and determining the pre-training weights of the classification model based on the component feature library.

[0014] In a second aspect, embodiments of this disclosure provide an electronic device, including: one or more processors; and a memory storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the interactive control method for digital modeling of historical buildings as described in the first aspect of this disclosure.

[0015] Thirdly, embodiments of this disclosure provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the interactive control method for digital modeling of historical buildings as described in the first aspect of this disclosure.

[0016] This disclosure provides an interactive control method for digital modeling of historical buildings, realizing a fusion framework modeling of three sources of data: laser point cloud, photogrammetry, and HBIM template library. It can automatically identify, classify, and bind attributes of components in three-dimensional space. Based on the established digital model of historical buildings, the story clues in the attribute information of the components are further structured and organized to establish knowledge graph nodes of the components. The components are associated with the story clues, and personalized explanatory information can be generated according to the user profile during user interaction. Taking the building components as the smallest explanatory unit, the explanatory content is dynamically generated to match the user's interests, language style, and cognitive level, truly realizing the intelligent and personalized "readable architecture". Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an interactive control method for digital modeling of historical buildings according to an embodiment of this disclosure.

[0018] Figure 2 This is a schematic diagram of the composition of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of this disclosure will be described in detail below with reference to the accompanying drawings.

[0020] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.

[0021] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0022] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0024] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0025] Firstly, referring to Figure 1 This disclosure provides an interactive control method for digital modeling of historical buildings, including: S1. Acquire point cloud data and images of the historical building, and establish a three-dimensional visualization model of the historical building, wherein the three-dimensional visualization model contains the geometric and texture information of the historical building; S2. Based on the geometric information and the texture information, identify the components of the historical building from the three-dimensional visualization model using a deep learning algorithm; S3. Match the component with the template components in the HBIM template library, wherein the HBIM template library includes a variety of template components and their geometric and texture information; S4. Based on the matching template component of the component, the attribute information of the component will be obtained from the archive database and bound to the component to obtain a digital model of the historical building; S5. Organize the story clues in the attribute information of the component in a structured manner and establish the knowledge graph nodes of the component. A component is associated with multiple story clues, and the story clues include at least one of the following: historical background, architectural style, character stories, and repair records. S6. In response to the user's first interactive instruction to click on the target component in the digital model of the historical building, based on the user profile and the knowledge graph node of the target component, call the large language model to generate personalized explanatory information.

[0026] In this embodiment of the disclosure, high-precision point cloud data of historical buildings is obtained by laser scanning. The point cloud data contains geometric information such as the overall and / or partial shape and size of the historical buildings.

[0027] In this embodiment of the disclosure, a high-resolution image of the exterior of a historical building is acquired using a photographic device, and the image contains overall and partial texture information of the historical building.

[0028] In this embodiment of the disclosure, the geometric information in the point cloud data is fused with the texture information in the image. This can utilize the high precision of the point cloud data and the rich and realistic information of the image to provide a rich information foundation for subsequent component recognition, thereby improving the accuracy of component recognition and facilitating automated modeling.

[0029] In this embodiment of the disclosure, various components of the historical building, such as windows, doors, gables, eaves, and gable walls, are identified from a three-dimensional visualization model using a deep learning algorithm.

[0030] Heritage Building Information Modelling (HBIM) is a method that applies Building Information Modeling (BIM) technology specifically to the digital protection, recording, analysis, restoration, and management of historical buildings, cultural heritage sites, and archaeological sites.

[0031] In this embodiment of the disclosure, an HBIM template library is established based on HBIM. The HBIM template library includes a variety of template components and their geometric and texture information. It is fused with point cloud data and images, breaking through the limitations of a single data source. By matching the identified components with the template components in the HBIM template library, the component displacement identifier (ID) is assigned.

[0032] In some embodiments, the HBIM template library is built based on characteristic components of historical buildings. For example, it covers various Shikumen architectural styles, including the geometric and material characteristics of typical components such as old-style Shikumen, new-style Shikumen, and garden-style lane houses. Through shape similarity calculation and texture feature comparison, a corresponding template component can be automatically matched for each identified component, and a unique component ID can be assigned to it.

[0033] In this embodiment of the disclosure, an archive database of historical buildings is also established. The archive database stores attribute information of each component of the historical buildings and distinguishes them with identifiers.

[0034] In some embodiments, attribute information, including material, style, repair time, damage level, and historical change records, is automatically bound to the corresponding component in the model, achieving true "component-level semantic modeling".

[0035] The embodiments of this disclosure define the following modeling process: a three-dimensional visualization model is established based on the point cloud data and images of the historical building; components in the three-dimensional visualization model are identified; the identifiers of the components are determined by matching with the HBIM template library; the attribute information of the components is obtained from the archive database through the identifiers and bound, thereby obtaining a digital model of the historical building.

[0036] The digital models of historical buildings established through the above process not only have visualization functions, but also endow the digital models with rich semantic information, which is conducive to meeting in-depth management, repair and display needs.

[0037] The entire process described above, from data collection to semantic binding, is automated, greatly reducing human intervention and improving modeling efficiency and information accuracy.

[0038] The digital model obtained by the interactive control method for digital modeling of historical buildings provided in this disclosure can be seamlessly integrated into 3D visualization platforms such as WebGL or Cesium, supporting online browsing, interactive querying and dynamic updates, and is suitable for various application scenarios such as building management, renovation decision-making, and public display.

[0039] In this embodiment of the disclosure, a digital model of a historical building based on its components supports the generation of component-level stories for public interaction, aiming to provide a personalized explanation experience where users click on components and the system instantly generates stories.

[0040] When a user clicks on a component in the digital model of a historical building, the system first identifies the user's interaction intent (such as "want to know the style of this entrance" or "want to know why this window is special"). Then, based on the user profile (such as age, language, and interest tags) and the knowledge graph nodes of the target component, it calls upon a large language model to generate a personalized explanatory message. This personalized explanation is not only accurate and vividly worded, but can also be combined with historical photos, before-and-after restoration comparison images, and audio readings to enhance the user experience.

[0041] In this embodiment of the disclosure, personalized narration information can be text information, image information, video information, or narration information in VR or AR format. This embodiment of the disclosure does not impose any special limitations on this.

[0042] In some embodiments, based on the geometric information and the texture information, identifying components of the historical building from the 3D visualization model using a deep learning algorithm includes: The point cloud of the three-dimensional visualization model is segmented using a deep learning algorithm to extract the geometric shape information of the components of the historical building. The components are classified using a deep learning algorithm to determine their texture feature information; Using a component classification model, target detection and style recognition are performed based on the geometric shape information and texture feature information of the component to determine the type and style information of the component.

[0043] In some embodiments, based on the user profile and the knowledge graph nodes of the target component, a large language model is invoked to generate personalized explanation information, including: Acquire user information, identify user interaction intent, and generate user profiles; For structured stories, a preset story template is invoked, and personalized explanation information is generated based on the user profile. For unstructured stories, a large language model is invoked to generate personalized explanation information based on the user profile.

[0044] In this embodiment, a hybrid approach of "rule templates + large language model" is adopted. For stories with clear structure and stable content (such as construction dates and style analysis), the system prioritizes the use of preset templates to ensure content accuracy; for open or personalized questions raised by users, the system calls the large language model for real-time generation, improving flexibility and naturalness.

[0045] In some embodiments, the attribute information of the component includes at least one of material, style, state, and event.

[0046] In some embodiments, events include the construction, expropriation, repair, maintenance, inspection, and operation of historical buildings.

[0047] In some embodiments, the interactive control method for digital modeling of historical buildings further includes: The lifecycle events corresponding to the historical buildings are obtained from the archive database, and parsed on a component-by-component basis to obtain the event information of the components. The event information of the components includes at least one of the following: time, location, participants, event description, and status comparison. An event data model is established, which represents the event information from the dimensions of time, space, and content, and associates the event with the component. In response to a second interactive instruction from a user to click on a target component in the digital model of the historical building, the events associated with the target component are visualized.

[0048] In some embodiments, it also includes: Acquire an event image, identify the event image, extract event information from the event image, and identify components in the event image; The extracted event information is updated in the event data model and associated with the identified components.

[0049] In some embodiments, the interactive control method for digital modeling of historical buildings further includes: Collect event data of the historical building, wherein the event data includes data generated from the repair, renovation and maintenance of the historical building; Identify event records of different components from the event data; The event records are bound to the associated components to update the digital model of the historical building.

[0050] This disclosure proposes a method for spatiotemporally coupling the entire lifecycle events of historical buildings with 3D model components, aiming to achieve a dynamic digital expression of "building as event, event as component." In the embodiments of this disclosure, an event database covering the entire process of building history, acquisition, construction, maintenance, and operation is constructed, with "event" as the core organizational unit. Each event not only includes basic information such as time, location, participants, and event description, but also, through spatial positioning and component binding mechanisms, is precisely associated with specific components or areas in the 3D model. For example, a facade renovation event not only records the renovation time and construction unit, but also binds it to all the components being repaired (such as doorways, window frames, and walls), and marks their specific locations in the model, comparing their states before and after the renovation.

[0051] In this embodiment, a unified event data model is first established, employing an "event-component-archive" triple structure for information organization. Here, "event" represents the time dimension, "component" represents the spatial dimension, and "archive" represents the source of information content. These three are linked through component identifiers, forming a scalable, queryable, and visualizeable knowledge network.

[0052] This disclosure does not impose any special limitations on how event data of historical buildings are collected.

[0053] For example, embodiments of this disclosure support access to multiple data sources, including historical documents, construction drawings, property inspection records, photos and videos, and oral interviews; they also support the automatic extraction of key event information through technologies such as OCR recognition, natural language processing, and image recognition, and its structured storage. For newly occurring events, such as defects discovered during property inspections, on-site photos can be taken and uploaded, the defect type and location can be automatically identified, and an event record can be generated and bound to the corresponding component, achieving "instant archiving."

[0054] In some embodiments, it also includes: In response to an instruction to generate a target component in the digital model of the historical building, the attribute information bound to the target component is obtained; The attribute information of the target is visualized.

[0055] In this embodiment of the disclosure, the final generated digital model of the historical building not only has a high-precision geometric structure and realistic visual representation, but also carries rich semantic information. Users can click on any component in the model to view its attribute cards and historical archives in real time, realizing the digital management concept of "model as archive".

[0056] The digital model of historical buildings provided in this disclosure also supports the visualization of events.

[0057] In some embodiments, the attribute information of the target is visualized, including: The event records of the target component are visualized in the form of a timeline; Based on the event records of the components, components that experienced events during the same period are highlighted in the digital model of the historical building; Based on the event records of the component, the changes of the same component at different times are compared and displayed.

[0058] For example, the digital model of historical buildings provided in this disclosure supports a two-dimensional interactive interface of "time and space". Users can dynamically view the model status and event distribution of the building in different historical periods by dragging the timeline. The digital model of historical buildings provided in this disclosure also supports automatic switching of model status, for example, switching from the architectural layout of a specific historical period to the current state after renovation, while highlighting specific components where events occurred during this period. For the status changes of the same component at different times, the digital model of historical buildings provided in this disclosure also supports comparison views to help users intuitively understand the evolution process of the building.

[0059] By spatiotemporally coupling events with digital models of historical buildings, this disclosed embodiment breaks through the limitations of traditional building archives being "static, isolated, and unstructured," achieving a unified expression of building lifecycle information in both time and space dimensions. It not only improves the efficiency of organizing historical building information and the ease of retrieval but also provides strong data support for the scientific management, precise restoration, and public education of historical buildings. It offers a feasible path to achieving the digital protection goal of "readable architecture, traceable history, and decision-making in restoration," possessing promising prospects for promotion and a leading role in the industry.

[0060] In some embodiments, classifying the components using a deep learning algorithm to determine the texture information of the components includes: Acquire images of sample components and annotate the images of the sample components; Based on the YOLOv8 framework, target detection is performed on the sample components in the sample component image, and the bounding boxes and category labels of the sample components are extracted. The CLIP (Contrastive Language-Image Pre-training) model aligns the image features within the bounding boxes of the sample components with the category labels to train the classification model. The components are classified using a trained classification model to determine their texture information.

[0061] Training the classification model through the above process not only improves its ability to recognize complex component styles, but also enhances its granular understanding of stylistic semantics (such as Baroque-style doorways, Roman columns, and Classical pediments). For example, the classification model can not only identify doorways, but also further determine whether they are traditional Chinese doorways or a blend of Chinese and Western Baroque styles.

[0062] In some embodiments, to enhance the model's generalization ability, the platform also introduces data augmentation and transfer learning strategies. During training, random rotation, occlusion, and color perturbation are used to expand the diversity of sample components, improving the classification model's adaptability to complex component shapes. Simultaneously, transfer training is performed using images of similar historical buildings collected from different regions and building complexes, ensuring the classification model maintains good recognition performance across different regions and building complexes.

[0063] In some embodiments, it also includes: The historical archive data of the historical building is analyzed, feature templates are extracted, and a component feature library is established. The feature templates represent location information, layout information, component information, etc., and the component feature library includes multiple two-dimensional vector contours and three-dimensional point cloud slices with labels of age, style, material, and craftsmanship. The pre-training weights of the classification model are determined based on the component feature library.

[0064] In this embodiment of the disclosure, historical archive data includes original SketchUp, CAD, Excel, and paper detail drawings of historical buildings.

[0065] In this embodiment, two-dimensional vector contours and three-dimensional point cloud slices labeled with era, style, material, and process can be directly used as pre-training weights for YOLOv8, thereby improving the first-round recognition accuracy of the classification model and solving the problem that general models cannot understand and recognize historical building components.

[0066] Secondly, referring to Figure 2 This disclosure provides an electronic device, which includes: One or more processors 101; The memory 102 stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the interactive control method for digital modeling of historical buildings as described in the first aspect of the present disclosure. One or more I / O interfaces 103 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0067] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, enabling information exchange between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0068] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0069] Thirdly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the interactive control method for digital modeling of historical buildings as described in the first aspect of this disclosure.

[0070] Fourthly, embodiments of this disclosure provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the interactive control method for digital modeling of historical buildings as described in the first aspect of this disclosure.

[0071] To enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of this disclosure, the technical solutions provided by the embodiments of this disclosure will be described in detail below through specific embodiments: Example 1 This embodiment proposes an automatic semantic annotation method for historical building components, integrating laser point cloud, photogrammetry, and HBIM template library, aiming to transform from "visible" 3D models to "understandable" intelligent models. First, high-precision point cloud data of the building is acquired through laser scanning, while high-resolution images of the building's exterior are collected using UAVs or ground-based photography equipment to construct a photogrammetry model. The two types of data complement each other geometrically and texturally, providing a rich information foundation for subsequent component identification. Subsequently, the platform employs a deep learning-based point cloud segmentation algorithm to extract component-level geometry from the building model, identifying representative facade components such as windows, doors, pediments, eaves, and gable walls. Simultaneously, the system performs object detection and style recognition on the texture images from the photogrammetry model, and, combined with a trained component classification model, automatically determines the type and style characteristics of the components.

[0072] After component identification, the system matches the extracted components with a pre-set HBIM template library. This template library is built based on characteristic components of a historical building, covering various architectural styles, including the geometric shapes and material characteristics of typical components such as old-style Shikumen, new-style Shikumen, and garden-style lane houses. Through shape similarity calculation and texture feature comparison, the system can automatically match the corresponding template for each identified component and assign it a unique component ID. Subsequently, the platform retrieves the relevant attribute information of the component from the archival database, including material, style, repair time, damage level, and historical change records, and automatically binds it to the corresponding component in the model, achieving true "component-level semantic modeling".

[0073] The entire process, from data acquisition to semantic binding, is automated, greatly reducing manual intervention and improving modeling efficiency and information accuracy. The final generated 3D model not only possesses high-precision geometric structure and realistic visual representation but also carries rich semantic information. Users can click on any component in the model to view its attribute cards and historical archives in real time, realizing the digital management concept of "model as archive." Based on the WebGL standard, this model can be seamlessly integrated into 3D visualization frameworks such as Cesium, supporting online browsing, interactive queries, and dynamic updates. It is suitable for various application scenarios such as building management, renovation decision-making, and public display.

[0074] This embodiment is significantly innovative in the following aspects: First, it proposes and implements for the first time a fusion framework of three data sources: laser point cloud, photogrammetric modeling, and HBIM template library, breaking through the limitations of traditional single data sources; second, by introducing deep learning algorithms, it achieves automatic identification and classification of building components in three-dimensional space, eliminating the arduous work of traditional manual modeling; third, it constructs a reusable component template library, enabling projects of similar architectural styles to be quickly migrated and applied, significantly improving the scalability and promotional value of the technology; finally, it achieves full-process automation from component identification to attribute binding, truly realizing "identification equals data entry," providing an efficient, accurate, and replicable technical path for the digital protection of historical buildings.

[0075] The interactive control method for digital modeling of historical buildings proposed in this embodiment not only achieves deep integration of 3D modeling and historical building archives in terms of technology, but also demonstrates high efficiency and practicality in engineering applications, possessing good prospects for promotion and industrial value; it provides a new technical path for the digital protection of historical buildings, and lays a solid data foundation for subsequent intelligent management, precise restoration and cultural dissemination.

[0076] Example 2 This embodiment proposes a method for generating stories at the component level of historical buildings, oriented towards public interaction intentions. The aim is to achieve a personalized explanation experience where "the system instantly generates a story when the user clicks on a component." This method uses architectural components as the smallest explanation unit, combining knowledge graphs, large language models, and rule templates to dynamically generate explanation content that matches the user's interests, language style, and cognitive level, truly realizing the intelligent and personalized "readable architecture."

[0077] In terms of system architecture, the platform first constructs a "component-level knowledge graph," which structurally organizes distinctive architectural components (such as Shikumen door lintels, window frames, gables, and brick carvings) with their historical background, architectural style, personal stories, and repair records. Each component is associated with multiple story threads, such as "construction stories," "repair events," "style analysis," and "folk tales," forming a rich content library. Simultaneously, the system supports multilingual and multi-style expressions, such as children's versions, expert versions, and youth versions, to meet the needs of different audiences.

[0078] When a user clicks on a component in the 3D model, the system first identifies their interaction intent (e.g., "Want to know the style of this entrance" or "Want to know why this window is special"). Then, based on the user profile (e.g., age, language, interest tags) and the knowledge graph nodes of the current component, it uses a large language model to generate a personalized explanatory text. This text is not only accurate and vivid, but can also incorporate historical photos, before-and-after restoration comparison images, and audio readings to enhance the user experience. For example, when a visitor clicks on the Baroque-style entrance of a historical building, the system can generate the following explanation: "The entrance you see is a Baroque-style design commissioned by a French designer for a historical building in the 1920s. Its carvings are inspired by the decorative patterns of the Louvre Museum and are cleverly integrated with the local blue and red brick mosaic technique. During the recent renovation, the craftsmen hand-cleaned each brick, preserving the original colors from a century ago. You can carefully observe the swirling patterns on the entrance, which symbolize 'welcome' and 'protection'." In addition, the system supports a "continuous interaction" function. If a user continues to ask, "How was this entrance renovated?" the system can generate more in-depth content about the renovation techniques, material sources, and construction difficulties based on the previous context, forming a multi-round dialogue-style explanation. This interactive method not only enhances visitor participation but also makes the architectural story more vivid and three-dimensional.

[0079] In terms of technical implementation, the platform adopts a hybrid approach of "rule templates + large language models". For stories with clear structures and stable content (such as construction dates and style analysis), the system prioritizes the use of preset templates to ensure accuracy. For open-ended or personalized questions raised by users, the system uses large language models for real-time generation, improving flexibility and naturalness. At the same time, the system introduces a content review mechanism to ensure that the generated content conforms to historical and cultural authenticity and avoids misrepresentation or fabrication.

[0080] The component-level story generation method proposed on this platform breaks through the limitations of traditional explanations that are "static, uniform, and non-interactive," and for the first time realizes an intelligent explanation mode that is "user-driven, component-responsive, and dynamically generated." This method not only enhances the public's understanding and interest in historical buildings, but also provides a new technological path for cultural tourism guidance, and has broad applicability and promotional value.

[0081] In conclusion, the historical building component-level story generation method proposed by this platform, which is geared towards public interactive intent, not only achieves the technical integration of knowledge graphs and large language models, but also demonstrates strong vitality and scalability in cultural dissemination practices. This method provides a new means of expression for the "revitalization and utilization" of historical buildings, and also provides solid technical support for building a cultural heritage dissemination system that is "participatory, perceptible, and resonant," possessing significant social value and promising prospects for promotion.

[0082] Example 3 In the process of protecting and managing historical buildings, the recording of building information is often fragmented and phased. From the building's historical evolution, acquisition process, construction and repair, to daily property inspections and operation, information at each stage is usually recorded separately by different departments and systems, lacking a unified organizational method and correlation mechanism. This information silo phenomenon makes it difficult for managers to fully grasp the true state of the building, and also brings great difficulties to subsequent repair decisions, accountability, and value interpretation.

[0083] This embodiment proposes a method for spatiotemporally coupling the entire lifecycle events of historical buildings with 3D model components, aiming to achieve a dynamic digital expression of "building as event, event as component." This method uses "event" as the core organizational unit to construct an event database covering the entire process of building history, acquisition, construction, maintenance, and operation. Each event not only includes basic information such as time, location, participants, and event description, but also, through spatial positioning and component binding mechanisms, is precisely associated with specific components or areas in the 3D model. For example, a facade renovation event not only records the renovation time and construction unit, but also binds it to all the components being repaired (such as doorways, window frames, and walls), and marks their specific locations in the model, comparing their states before and after the renovation.

[0084] This embodiment establishes a unified event data model, employing an "event-component-archive" ternary structure for information organization. In this structure, "event" represents the time dimension, "component" represents the spatial dimension, and "archive" represents the source of information content. These three elements are linked through unique identifiers, forming a scalable, queryable, and visualizeable knowledge network. This embodiment supports users browsing the historical evolution of a building through various methods such as timelines, component trees, or map views. For example, a user can click on a component in the 3D model, and the system will automatically display all relevant events for that component since the building's construction, including repairs, damage records, and renovations, presenting its status change trajectory in timeline form.

[0085] Regarding the collection and entry of event data, this embodiment supports access to multiple data sources, including historical documents, construction drawings, property inspection records, photos and videos, and oral interviews. This embodiment supports the automatic extraction of key event information through technologies such as OCR recognition, natural language processing, and image recognition, and its structured storage. For newly occurring events, such as defects discovered during property inspections, on-site photos can be uploaded, the defect type and location can be automatically identified, and an event record can be generated and bound to the corresponding component, achieving "instant archiving."

[0086] In terms of visualization, this embodiment provides a two-dimensional interactive interface of "time and space". Users can dynamically view the model status and event distribution of a building in different historical periods by dragging the timeline. The system supports automatic switching of model status. For example, for a historical building, the system can switch from the architectural layout of a specific historical period to its current state after renovation, while highlighting specific components that experienced events during this period. For the same component's status changes at different times, the system also supports comparative views to help users intuitively understand the building's evolution.

[0087] The spatiotemporal coupling method proposed in this embodiment breaks through the limitations of traditional building archives, which are "static, isolated, and unstructured." For the first time, it incorporates "events" as first-class citizens into the three-dimensional model system, achieving a unified expression of building lifecycle information in both time and space dimensions. This method not only improves the efficiency of building information organization and the convenience of querying, but also provides strong data support for the scientific management, precise repair, and public education of buildings.

[0088] The spatiotemporal coupling method of full life-cycle events and three-dimensional components proposed in this embodiment not only achieves dynamic organization and visualization of building information technically, but also demonstrates extremely high application value in management practice. This method provides systematic data support for the protection and management of historical buildings, and also offers a feasible path to achieving the digital protection goal of "readable architecture, traceable history, and decision-making for repairs," possessing good prospects for promotion and a leading role in the industry.

[0089] Example 4 A certain historical building is a typical representative of modern urban residences in China. Its architectural style blends traditional Jiangnan dwellings with Western decorative art elements, resulting in unique details such as doorways, window frames, gables, and brick carvings. These components not only possess extremely high artistic value but also serve as important evidence for determining the building's age, style, and historical significance. However, due to the sheer number, complex styles, and varied details of Shikumen buildings, traditional manual identification and recording methods are inefficient and highly subjective, making it difficult to meet the needs of large-scale digital archiving and intelligent management.

[0090] This embodiment proposes a component recognition model training method for Shikumen architectural styles, aiming to build a high-precision recognition system specifically for Shikumen buildings, enabling automatic detection, type determination, and style classification of typical components. This method not only improves the accuracy and adaptability of recognition but also provides a technical foundation for subsequent architectural style analysis, component semantic annotation, and automatic archive generation.

[0091] During the data collection phase, the platform focused on a specific historical building area, collecting image data covering 43 Shikumen buildings, including key components such as facades, doorways, window frames, gables, brick carvings, and balconies. Image acquisition methods included drone aerial photography, DSLR camera spot shooting, and mobile app-based patrol photography, ensuring the data's diversity and representativeness in terms of perspective, lighting, and resolution. All images were manually labeled, resulting in a Shikumen component image dataset containing over 10,000 images covering more than 20 component types.

[0092] For model training, YOLOv8 was used as the basic detection framework, combined with the CLIP (Contrastive Language-Image Pre-training) model for semantic enhancement. First, YOLOv8 was used to detect objects in the image, extracting their bounding boxes and category labels. Then, the CLIP model was used to align image features with textual descriptions (such as Baroque-style doorways, blue brick window frames, and red brick mosaics), achieving joint modeling of the image and semantic space. This approach not only improves the model's ability to recognize complex styles but also gives it a fine-grained understanding of style semantics. For example, the model can not only identify doorways but also further determine whether they are traditional Chinese doorways or a blend of Chinese and Western Baroque styles.

[0093] To enhance the model's generalization ability, this embodiment also introduces data augmentation and transfer learning strategies. During training, random rotation, occlusion, and color perturbation are used to expand sample diversity and improve the model's adaptability to complex environments. Simultaneously, transfer training is performed using images collected from other Shikumen areas, enabling the model to maintain good recognition performance in different blocks and building complexes.

[0094] At the application level, after users upload building photos or inspection images, the system can automatically identify the type and style of components in the image and recommend corresponding HBIM templates and attribute information, which greatly improves the efficiency and standardization of document creation.

[0095] The component recognition model training method for Shikumen architectural style proposed in this embodiment fills the current technical gap in the field of detailed component recognition of historical buildings, and realizes automatic detection and style classification of unique Shikumen components. It not only possesses high accuracy and strong generalization ability, but also provides a reusable technical tool for the large-scale digital and intelligent management of historical buildings, and has broad promotional value and industry influence.

[0096] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0097] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. An interactive control method for digital modeling of historical buildings, comprising: Obtain point cloud data and images of historical buildings, and establish a three-dimensional visualization model of the historical buildings. The three-dimensional visualization model contains the geometric and texture information of the historical buildings. Based on the geometric and texture information, components of the historical building are identified from the 3D visualization model using a deep learning algorithm; The component is matched with template components in the HBIM template library, wherein the HBIM template library includes a variety of template components and their geometric and texture information; Based on the matching template component of the component, the attribute information of the component will be obtained from the archive database and bound to the component to obtain a digital model of the historical building; The story clues in the attribute information of the component are structured and organized to establish the knowledge graph nodes of the component. A component is associated with multiple story clues, and the story clues include at least one of the following: historical background, architectural style, personal stories, and repair records. In response to the user's first interactive instruction to click on the target component in the digital model of the historical building, the system calls a large language model to generate personalized explanatory information based on the user profile and the knowledge graph nodes of the target component.

2. The interactive control method for digital modeling of historical buildings according to claim 1, wherein, Based on the geometric information and the texture information, components of the historical building are identified from the 3D visualization model using a deep learning algorithm, including: The point cloud of the three-dimensional visualization model is segmented using a deep learning algorithm to extract the geometric shape information of the components of the historical building. The components are classified using a deep learning algorithm to determine their texture feature information; Using a component classification model, target detection and style recognition are performed based on the geometric shape information and texture feature information of the component to determine the type and style information of the component.

3. The interactive control method for digital modeling of historical buildings according to claim 1, wherein, Based on the user profile and the knowledge graph nodes of the target component, a large language model is invoked to generate personalized explanation information, including: Acquire user information, identify user interaction intent, and generate user profiles; For structured stories, a preset story template is invoked, and personalized explanation information is generated based on the user profile. For unstructured stories, a large language model is invoked to generate personalized explanation information based on the user profile.

4. The interactive control method for digital modeling of historical buildings according to claim 1, wherein, The interactive control method for digital modeling of historical buildings also includes: The lifecycle events corresponding to the historical buildings are obtained from the archive database, and parsed on a component-by-component basis to obtain the event information of the components. The event information of the components includes at least one of the following: time, location, participants, event description, and status comparison. An event data model is established, which represents the event information from the dimensions of time, space, and content, and associates the event with the component. In response to a second interactive instruction from a user to click on a target component in the digital model of the historical building, the events associated with the target component are visualized.

5. The interactive control method for digital modeling of historical buildings according to claim 4, wherein, Also includes: Acquire an event image, identify the event image, extract event information from the event image, and identify components in the event image; The extracted event information is updated in the event data model and associated with the identified components.

6. The interactive control method for digital modeling of historical buildings according to claim 4, wherein, Visualize the events associated with the target component, including: The events associated with the target component are visualized in a timeline format; Based on the event information of the components, components that experienced events during the same period are highlighted in the digital model of the historical building; Based on the event information of the component, the changes of the same component at different times are compared and displayed.

7. The interactive control method for digital modeling of historical buildings according to claim 2, wherein, The components are classified using a deep learning algorithm to determine their texture feature information, including: Acquire images of sample components and annotate the images of the sample components; Based on the YOLOv8 framework, target detection is performed on the sample components in the sample component image, and the bounding boxes and category labels of the sample components are extracted. The CLIP model is used to align the image features in the bounding box of the sample component with the category label, thereby performing joint modeling of the image and semantic space to train the classification model. The components are classified using a trained classification model to determine their texture feature information.

8. The interactive control method for digital modeling of historical buildings according to claim 7, wherein, Also includes: The historical archive data of the historical building is analyzed, feature templates are extracted, and a component feature library is established. The feature templates represent location information, layout information, component information, etc., and the component feature library includes multiple two-dimensional vector contours and three-dimensional point cloud slices with labels of age, style, material, and craftsmanship. The pre-training weights of the classification model are determined based on the component feature library.

9. An electronic device, comprising: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the interactive control method for digital modeling of historical buildings according to any one of claims 1 to 8.

10. A computer program product comprising a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the interactive control method for digital modeling of historical buildings according to any one of claims 1 to 8.

Images