Traditional village protection method and device based on digital twin baseline

Abstract

This application discloses a method and apparatus for the protection of traditional villages based on digital twin baselines, comprising: acquiring multi-source spatial data and performing preprocessing and coordinate registration to obtain spatial baseline framework data and registered multi-source data; extracting features of protected buildings and identifying value elements to obtain protected building object data; constructing a spatial association graph and performing protection priority learning and zoning boundary generation to obtain protected zoning vector data; constructing a digital twin baseline model based on the protected building object data; performing cross-modal registration of monitoring video stream data and the digital twin baseline model; and performing building anomaly detection and construction activity monitoring based on the registered monitoring data and protected zoning vector data to obtain anomaly monitoring result data, and performing graded early warning and reliable evidence storage processing. This application improves the efficiency of protected building identification, can proactively detect building hazards and illegal construction activities, and prevent damage to the traditional village landscape.

Description

Technical Field

[0001] This application relates to the field of digital preservation technology for cultural heritage, and in particular to a method and apparatus for the preservation of traditional villages based on a digital twin baseline. Background Technology

[0002] Traditional villages are important spatial carriers of historical memories of agricultural civilization and local cultural genes. Their protection involves multiple aspects such as building identification, protection zoning, and dynamic monitoring, which are directly related to the integrity and continuity of the historical features of the villages. However, existing traditional village protection technologies have significant shortcomings in all aspects mentioned above: In terms of building identification, the general survey of protected buildings mainly relies on manual on-site inspections and expert judgment, which is labor-intensive and inefficient. Moreover, modern and traditional buildings are mixed together, making it difficult for manual identification to quickly distinguish them, resulting in a long cycle and insufficient coverage in the establishment of the protected building list; In terms of protection zoning, the delineation of the core protection area and construction control zone mainly relies on the subjective experience of planners, lacking objective and quantitative scientific basis. The zoning boundaries do not closely match the natural geographical boundaries, and the zoning results are presented in the form of static drawings, which are difficult to adapt to the needs of dynamic changes in villages; In terms of monitoring and early warning, the existing monitoring camera system only has passive recording function and lacks linkage with protected building information and protection zoning data, making it impossible to proactively detect building hazards and illegal construction activities, and the early warning response is seriously delayed; More importantly, in the existing technology, the three links of building identification, zoning generation, and monitoring and early warning are independent systems with inconsistent data formats and information not being shared, making it impossible to form a closed-loop protection system of discovery, protection, and monitoring. As a result, the traditional village landscape continues to be damaged without effective technical support. Summary of the Invention

[0003] The main purpose of this application is to provide a traditional village protection method based on digital twin baselines, which aims to solve the technical problems of the current traditional village protection relying on manual surveys to identify protected buildings, which is inefficient, and the passive and lagging monitoring and early warning cannot actively detect building hazards and illegal construction activities that lead to the damage to the traditional village landscape.

[0004] To achieve the above objectives, this application proposes a method for the protection of traditional villages based on digital twin baselines. This method includes: Acquire multi-source spatial data and perform preprocessing and coordinate registration to obtain spatial reference frame data and registered multi-source data; Based on the spatial reference framework data and the registered multi-source data, the protected building features are extracted and the value elements are identified to obtain protected building object data. By integrating the data of the protected building objects, existing protected object data, and natural geographic element data, a spatial relationship map is constructed, and protection priority learning and zoning boundary generation are performed to obtain protected zoning vector data; A digital twin baseline model is constructed based on the data of the protected building object, wherein the digital twin baseline model includes a multi-level semantic model of the building appearance and a three-dimensional geometric benchmark model. Acquire surveillance video stream data, and perform cross-modal registration of the surveillance video stream data with the digital twin baseline model to obtain registered surveillance data; Based on the registered monitoring data and the protected area vector data, building anomaly detection and construction activity monitoring are carried out to obtain anomaly monitoring result data. Based on the anomaly monitoring result data, graded early warning and reliable evidence storage are carried out to achieve the protection and monitoring of traditional villages.

[0005] Furthermore, to achieve the above objectives, this application also proposes a traditional village protection device based on a digital twin baseline, which includes: The registration module is used to acquire multi-source spatial data and perform preprocessing and coordinate registration to obtain spatial reference frame data and registered multi-source data. The identification module is used to extract the features of protected buildings and identify their value elements based on the spatial reference frame data and the registered multi-source data, so as to obtain protected building object data. The construction module is used to integrate the data of the protected building objects, existing protected object data and natural geographic element data, construct a spatial relationship map, and perform protection priority learning and zoning boundary generation to obtain protected zoning vector data; The construction module is also used to construct a digital twin baseline model based on the protected building object data, wherein the digital twin baseline model includes a multi-level semantic model of the building appearance and a three-dimensional geometric benchmark model. The registration module is also used to acquire monitoring video stream data, and to perform cross-modal registration of the monitoring video stream data with the digital twin baseline model to obtain the registered monitoring data. The monitoring module is used to detect building anomalies and monitor construction activities based on the registered monitoring data and the protected area vector data, obtain anomaly monitoring result data, and perform graded early warning and reliable evidence storage processing based on the anomaly monitoring result data, so as to realize the protection and monitoring of traditional villages.

[0006] Furthermore, to achieve the above objectives, this application also proposes a traditional village protection device based on a digital twin baseline, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the traditional village protection method based on a digital twin baseline as described above.

[0007] In addition, to achieve the above objectives, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the traditional village protection method based on digital twin baselines as described above.

[0008] This application proposes one or more technical solutions, which involve acquiring multi-source spatial data and performing preprocessing and coordinate registration to obtain spatial reference framework data and registered multi-source data; extracting protected building features and identifying value elements based on the spatial reference framework data and the registered multi-source data to obtain protected building object data; fusing the protected building object data, existing protected object data, and natural geographic element data to construct a spatial correlation map and perform protection priority learning and zoning boundary generation to obtain protected zoning vector data; constructing a digital twin baseline model based on the protected building object data, wherein the digital twin baseline model includes a multi-level semantic model of building appearance and a three-dimensional geometric reference model; acquiring monitoring video stream data and performing cross-modal registration between the monitoring video stream data and the digital twin baseline model to obtain registered monitoring data; performing building anomaly detection and construction activity monitoring based on the registered monitoring data and the protected zoning vector data to obtain anomaly monitoring result data, and performing graded early warning and reliable evidence storage processing based on the anomaly monitoring result data to achieve the protection and monitoring of traditional villages. Through the above methods, by constructing a spatial correlation graph and using graph neural networks for protection priority learning, the objective and scientific generation of protection zoning boundaries has been achieved. Through cross-modal registration of digital twin baseline models and real-time monitoring videos, the accuracy of building hazard detection has been significantly improved and the false alarm rate has been reduced. Through the dynamic linkage between adaptive electronic fences and zoning results, data association and collaborative linkage between building identification, zoning generation and real-time monitoring and early warning in the protection of traditional villages have been realized. This has improved the efficiency of building identification, enabled the proactive detection of building hazards and illegal construction activities, enhanced the intelligence level and response time of traditional village protection, and prevented the destruction of the traditional village landscape. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating an embodiment of the traditional village protection method based on digital twin baselines provided in this application. Figure 2 This is a schematic diagram of the modular structure of a traditional village protection device based on a digital twin baseline, as described in an embodiment of this application.

[0012] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0014] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0015] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, or a traditional village protection device based on a digital twin baseline. The following description uses a traditional village protection device based on a digital twin baseline as an example to illustrate this embodiment and the subsequent embodiments.

[0016] Based on this, the embodiments of this application provide a method for the protection of traditional villages based on digital twin baselines, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the traditional village protection method based on digital twin baselines in this application.

[0017] In this embodiment, the traditional village protection method based on digital twin baselines includes steps S10~S60: Step S10: Acquire multi-source spatial data and perform preprocessing and coordinate registration to obtain spatial reference frame data and registered multi-source data.

[0018] It should be noted that multi-source spatial data refers to a collection of spatial data originating from different sensors, platforms, and resolutions, including at least remote sensing imagery, terrestrial imagery, and basic geographic information data. Remote sensing imagery refers to images of village areas acquired through aerial or space platforms, including satellite remote sensing imagery and UAV aerial imagery, used to obtain information on the overall spatial layout and building distribution of villages. Terrestrial imagery refers to images of building facades and details acquired through ground-based close-range photography equipment, used to capture detailed features of building exteriors. Basic geographic information data refers to vector or raster data describing the geospatial framework of villages, including at least digital elevation models and village boundary vector data, providing a unified spatial reference benchmark for the entire system.

[0019] Preprocessing refers to the data processing steps of error correction, format conversion, and quality improvement for raw multi-source spatial data. Coordinate registration refers to the geometric transformation process of unifying data from different sources and coordinate systems to the same spatial coordinate reference system. Spatial reference frame data refers to a three-dimensional spatial framework covering the entire village area with a unified coordinate reference, providing a benchmark for subsequent overlay analysis and correlation calculations of all spatial data. Registered multi-source data refers to the set of multi-source spatial data with a unified spatial coordinate reference after preprocessing and coordinate registration.

[0020] Understandably, the protection of traditional villages involves diverse data sources and significant differences in scale. Satellite imagery typically has sub-meter resolution, drone aerial imagery can reach centimeter resolution, and ground imagery has even higher resolution. Furthermore, the imaging geometry, radiometric characteristics, and coordinate systems of these three types of images differ. Without a unified spatial benchmark framework, subsequent building identification, zoning generation, and monitoring registration cannot achieve accurate spatial correlation.

[0021] In one feasible implementation, step S10 may include: acquiring multi-source spatial data, wherein the multi-source spatial data includes at least remote sensing image data, ground image data, and basic geographic information data; performing radiometric and geometric correction processing on the remote sensing image data, performing distortion correction and illumination enhancement processing on the ground image data, and performing coordinate system unification and format standardization processing on the basic geographic information data to obtain preprocessed multi-source data; establishing a unified spatial coordinate reference system based on the preprocessed multi-source data to obtain spatial reference frame data; and performing spatial coordinate registration on the preprocessed multi-source data based on the spatial reference frame data to obtain registered multi-source data.

[0022] It should be noted that remote sensing image data refers to Earth surface image data acquired through sensors mounted on aerial or space platforms, including at least satellite remote sensing imagery and drone aerial imagery. Satellite remote sensing imagery is characterized by its wide coverage and short revisit period, making it suitable for acquiring information on the overall spatial layout of villages and macroscopic building distribution, with spatial resolution typically ranging from sub-meter to meter. Drone aerial imagery is characterized by its high resolution and flexible viewing angle, making it suitable for acquiring details of building roofs and textures of village streets and alleys, with spatial resolution reaching the centimeter level. Terrestrial imagery data refers to images of building facades and details acquired through ground-based close-range photography equipment, such as digital cameras, panoramic cameras, and mobile surveying vehicles. It has the highest spatial resolution and the richest texture details, making it suitable for capturing subtle features of building exteriors, such as brick patterns, roof tiles, and carvings.

[0023] Basic geographic information data refers to the fundamental data describing the geospatial framework of villages, including at least a digital elevation model (DEM) and village boundary vector data. The DEM is surface elevation data expressed in the form of a regular grid or irregular triangular network, used to describe the topographic relief and geomorphic features of villages, and serves as the foundation for subsequent 3D reconstruction, line-of-sight analysis, and zoning generation. Village boundary vector data is the administrative boundary or natural settlement boundary of a village expressed in polygon form, used to define the spatial scope of conservation work.

[0024] Understandably, the preservation of traditional villages involves diverse data sources, varying scales, and different coordinate systems. Satellite imagery typically uses the WGS-84 geographic coordinate system, while drone aerial imagery may use local independent coordinate systems, and ground imagery usually lacks geographic coordinate information. Without preprocessing and coordinate unification of this data, subsequent building identification, zoning generation, and monitoring registration cannot achieve accurate spatial correlation. This can even lead to discrepancies of tens of meters in the location of the same building across different data sources, resulting in serious consequences such as incorrect building positioning and zoning boundary shifts.

[0025] Radiometric correction is a process that eliminates or reduces the influence of sensor response characteristics, atmospheric transmission effects, and illumination conditions on the radiance values ​​of images. When sensors acquire images, factors such as differences in quantum efficiency, dark current noise, and gain shift lead to a systematic deviation between the original digital quantization value (DN) and the true surface radiance. Atmospheric transmission effects refer to the scattering and absorption of solar radiation as it passes through the atmosphere before reaching the Earth's surface, resulting in reduced image contrast and color distortion. Radiometric correction converts the original DN value into surface reflectance or radiance values ​​by establishing sensor calibration coefficients and an atmospheric radiative transmission model, ensuring comparable radiometric characteristics for images acquired at different times and by different sensors. Radiometric correction includes sensor calibration and atmospheric correction. Sensor calibration converts the original DN value into radiance values ​​at the sensor's entrance pupil based on the calibration parameters of satellite or UAV sensors, eliminating systematic errors introduced by the sensor itself. Atmospheric correction uses atmospheric correction methods based on radiative transmission models, such as the 6S model, MODTRAN model, or dark pixel method, to eliminate the effects of atmospheric scattering and absorption, thus obtaining the surface reflectance.

[0026] Geometric correction is a process that eliminates imaging geometric distortions and projects the image onto a specified map coordinate system. Imaging geometric distortions include system distortion, which is internal distortion caused by sensor imaging geometry, and terrain distortion, which is external distortion caused by surface undulations. System distortions include image point displacements caused by nonlinear motion of the scanning mirror, detector alignment errors, focal length deviations, etc. Terrain distortions refer to the shift in the projected position of image points on different elevation surfaces due to changes in surface elevation, which is particularly noticeable in traditional villages in mountainous or hilly areas. Geometric correction includes system geometric correction and terrain geometric correction. System geometric correction establishes a rational polynomial coefficient model or a rigorous physical imaging model based on the orbital parameters, attitude parameters, and sensor imaging geometric model of the satellite or UAV platform. It establishes a mapping relationship between the original image pixel coordinates (r, c) and geographic coordinates (lat, lon). By selecting uniformly distributed ground control points with coordinate accuracy requirements, the model parameters are corrected to achieve system distortion correction of the image. Terrain geometric correction, on the other hand, uses an orthorectification method based on collinearity equations based on the Digital Elevation Model (DEM) to eliminate image point displacement caused by terrain undulations. For each pixel, the accurate ground coordinates (X, Y) are iteratively solved based on the DEM elevation value Z and the RPC model to generate an orthorectified image. The image obtained through orthorectification can be directly matched and overlaid with basic geographic information data, and the correction accuracy can meet the requirements of traditional large-scale spatial mapping of villages.

[0027] Distortion correction targets close-up ground images, primarily eliminating geometric distortions caused by lens distortion and tilted shooting angles. It obtains intrinsic parameters and distortion coefficients through camera calibration, corrects radial and tangential distortion, and then corrects perspective distortion caused by tilted shooting angles through projection transformation, restoring the geometric relationships of building facades in the image to reality.

[0028] Illumination enhancement is a process that improves the visual quality of images and highlights architectural texture features. Traditional village buildings are mostly made of blue bricks and gray tiles with carved wooden structures. Under different lighting conditions, such as cloudy days, backlighting, and sidelighting, the contrast and color saturation of the images vary significantly, affecting the accuracy of subsequent feature extraction and recognition. Illumination enhancement adaptively adjusts the brightness, contrast, and color balance of images to ensure that architectural texture features remain consistently discernible under different lighting conditions. Illumination enhancement processing includes brightness equalization, contrast enhancement, and color correction, which can effectively improve the accuracy of subsequent feature extraction and building recognition.

[0029] Coordinate System 1 for basic geographic information data is a process that transforms geographic information data from different sources and with different coordinate reference systems to the same coordinate reference system. The basic geographic information data involved in traditional village preservation work may originate from different surveying projects and data production units, employing different coordinate systems. Coordinate System 1 uses a seven-parameter Bursa model or a four-parameter similarity transformation model to transform coordinates from different coordinate systems to a unified target coordinate system.

[0030] Format standardization is a process that converts geographic information data in different formats into a unified exchange format. Format standardization uses data format conversion tools to transform all data into a unified format and standardize attribute field naming, encoding methods, and metadata standards to ensure compatibility for subsequent data fusion and analysis. Based on the preprocessed multi-source data, a unified spatial coordinate reference system is established to obtain spatial benchmark framework data. The spatial benchmark framework data is a three-dimensional spatial framework covering the entire village area with a unified coordinate reference, providing a benchmark for the overlay analysis and correlation calculation of all subsequent spatial data. The spatial benchmark framework data includes plane coordinate benchmarks, elevation benchmarks, time benchmarks, and spatial resolution benchmarks.

[0031] Spatial coordinate registration refers to the geometric transformation process of unifying data from different sources, with different resolutions and imaging geometries, into a single spatial coordinate reference system. Registered multi-source data have a unified spatial coordinate reference, ensuring that the coordinates of any spatial location are consistent across different data sources.

[0032] In a preferred embodiment, to address resolution differences and geometric distortions among multi-source data, spatial coordinate registration is performed on the preprocessed multi-source data based on the spatial reference frame data to obtain registered multi-source data. This can be achieved using a multi-level image registration strategy from coarse to fine, specifically including: acquiring satellite imagery and UAV aerial imagery from the remote sensing imagery data; determining the resolution difference ratio between the satellite imagery and the UAV aerial imagery; based on the resolution difference ratio, employing a multi-level image registration strategy from coarse to fine, first performing global affine transformation registration on the UAV aerial imagery using the satellite imagery as a reference to obtain coarsely registered aerial imagery data; and then, based on the coarsely registered aerial imagery data... The aerial image data is processed by extracting building corner features and texture features from the UAV aerial images, and performing local fine-tuning registration based on these features to obtain finely registered aerial image data. Based on the finely registered aerial image data, a set of ground control points is extracted from the ground image data, including building facade feature points and road intersection feature points. According to the set of ground control points, a photogrammetric resection method based on collinearity equations is used to determine the exterior orientation elements of the ground image data. Based on the exterior orientation elements and the spatial reference frame data, the ground image data is projected onto a unified spatial coordinate reference system to obtain registered multi-source data.

[0033] It should be noted that the coarse-to-fine multi-level image registration strategy is a layered registration method that first performs global coarse alignment and then performs local fine adjustments, effectively handling the registration difficulties caused by large-scale resolution differences. Global affine transformation registration is an image registration method based on a six-parameter affine transformation model, capable of translation, rotation, scaling, and shearing transformations, suitable for coarse-scale alignment. Local fine-tuning registration is a registration method that uses local image features for minor geometric corrections based on global registration, eliminating local distortions that global transformation cannot handle. The collinearity equation is a fundamental equation in photogrammetry describing the collinear relationship between the camera center, image points, and ground points. Resection is a method for solving the spatial position and attitude parameters of the camera center based on known ground control points. Exterior orientation elements are six parameters describing the three-dimensional coordinates and three attitude angles of the camera center in the object space coordinate system, namely pitch, yaw, and roll angles.

[0034] Understandably, traditional villages are characterized by dense buildings and narrow streets, resulting in significant geometric differences and occlusion between ground and aerial imagery. Direct global registration is insufficient to achieve pixel-level accuracy. However, a three-level registration strategy, from coarse to fine, such as satellite → aerial → ground, can gradually reduce registration errors. Ultimately, photogrammetric resection accurately incorporates high-resolution ground imagery into a unified spatial framework, providing a high-precision data foundation for subsequent fine-grained building identification.

[0035] In actual implementation, the resolution difference ratio R between satellite imagery and drone aerial imagery is first determined as R=R satellite / R uav , where R satellite For satellite imagery ground resolution, R uav The ground resolution of the UAV aerial imagery is determined. Using satellite imagery as a reference, a feature matching method based on scale-invariant feature transform (SIFT) is employed to determine the global affine transformation matrix H of the UAV aerial imagery. global Coarse registration is performed on the drone aerial imagery. From the coarsely registered aerial imagery, building corner features, such as eaves corners and ridge endpoints, and texture features, such as wall brick patterns and tile textures, are extracted. A local refinement transformation matrix H is determined using a local template matching method based on normalized cross-correlation. local This yielded precisely registered aerial images.

[0036] Based on the precisely registered aerial imagery, ground control points are manually or automatically selected from the terrestrial imagery. Ground control points should meet the following conditions: clearly visible in both aerial and terrestrial imagery; possessing distinct geometric features, such as the corner points of doors and windows on building facades or the corner points of curb stones at road intersections; and being evenly distributed, covering the main areas of the village. Each terrestrial image requires at least six ground control points. The correspondence between image point coordinates and terrestrial point coordinates is established using collinearity equations.

[0037] Where (x,y) are the image point coordinates, (x0,y0) are the principal image point coordinates, f is the camera principal distance, and (X,Y,Z) are the ground point coordinates. S ,Y S Z S ) represents the coordinates of the photography center, a i ,b i ,c i (i=1,2,3) are the elements of the rotation matrix consisting of three attitude angles.

[0038] Solving the above system of equations using the least squares method yields the exterior orientation elements (X) of the ground image. S ,Y S Z S ,φ,ω,κ), where φ is the pitch angle, ω is the yaw angle, and κ is the roll angle. Based on the obtained exterior orientation elements and coordinate reference system parameters in the spatial reference frame data, the ground image is projected onto a unified spatial coordinate reference system to complete the spatial coordinate registration of multi-source data.

[0039] Step S20: Based on the spatial reference frame data and the registered multi-source data, extract the features of the protected buildings and identify the value elements to obtain protected building object data.

[0040] It should be noted that protected building feature extraction refers to the process of extracting characteristic parameters that can distinguish between traditional and modern buildings from registered multi-source data. Value element identification refers to the process of conducting fine-grained analysis on the initially screened buildings to identify whether they have historical, artistic, scientific, or other conservation value. Protected building object data refers to the set of identified and confirmed protected buildings, including the unique identifier, spatial location, building attributes, and value element information of each protected building.

[0041] Understandably, modern and traditional buildings are mixed in traditional villages. Modern buildings mostly have flat roofs or corrugated steel roofs, while traditional buildings mostly have pitched roofs and gray tile roofs. This difference in form and color is a key basis for rapid initial screening. However, relying solely on roof features for initial screening carries the risk of misjudgment, as some modern imitation antique buildings may also have pitched roofs and gray tile appearances. Therefore, secondary confirmation is needed through fine-grained value element identification.

[0042] In one feasible implementation, step S20 may include: extracting morphological and color feature parameters of building roofs based on the registered multi-source data; jointly screening the morphological and color feature parameters according to a preset protected building feature threshold to obtain preliminary building set data; extracting local building image data from the registered multi-source data for each building in the preliminary building set data; performing value element detection using a fine-grained visual recognition model based on the local building image data to obtain value element detection result data; determining value based on the value element detection result data, identifying buildings with value elements as protected buildings, and determining the spatial location information of each protected building based on the spatial reference frame data to obtain protected building object data.

[0043] It should be noted that the morphological characteristics of a building roof are quantitative indicators describing its geometry, including at least the roof slope and the geometric features of its outline. The roof slope refers to the angle between the roof slope and the horizontal plane, usually expressed as an angle or percentage. Traditional pitched roofs typically have slopes between 20° and 45°, with the specific value varying depending on the regional climate and building type. For example, slopes are steeper in rainy southern regions and gentler in drier northern regions. Modern flat roofs usually have slopes less than 5°, while corrugated steel roofs have a wider range of slope values ​​but a more regular outline. The geometric features of the roof outline refer to the shape characteristics of the roof on its horizontal projection plane, including the direction of the ridge line, the complexity of the eaves outline, and the roof's plan shape, such as a gable roof, hip roof, or overhanging gable roof.

[0044] Color characteristic parameters are quantitative indicators describing the color characteristics of building roofs, including at least the roof hue characteristic value and the roof saturation characteristic value. The roof hue characteristic value refers to the H component value of the roof color in the HSV (Hue-Saturation-Lightness) color space. Traditional gray tiles typically have hue values ​​in the low-saturation gray region, meaning the H value is around 0° or 180°, depending on the color space definition. Modern color steel tiles have a wider range of hue values, such as red, blue, and green. The roof saturation characteristic value refers to the purity index of the roof color, i.e., the S component. Due to weathering and material properties, traditional gray tiles usually have lower saturation values ​​(S < 0.3), while modern color steel tiles typically have higher saturation values ​​(S > 0.5).

[0045] Understandably, modern and traditional buildings are interspersed in traditional villages. Modern buildings mostly have flat roofs or corrugated steel roofs, while traditional buildings mostly have pitched roofs and gray tile roofs. This difference in form and color is a key basis for rapid initial screening. However, relying solely on a single feature for initial screening carries the risk of misjudgment: some modern imitation antique buildings may have a pitched roof appearance but lack traditional value elements; some traditional buildings may have undergone modern renovations, such as replacing the corrugated steel roof, but still have conservation value. Therefore, a combined form-color screening, along with the identification of fine-grained value elements, is needed for secondary confirmation.

[0046] The preset thresholds for protected building characteristics are criteria determined based on the regional architectural features of the target villages and expert experience. Because traditional architecture varies significantly across different regions—for example, Lingnan-style wok-ear houses, Hui-style horse-head walls, and northern courtyard houses—the preset thresholds need to be adaptively adjusted according to the specific architectural characteristics of each village. Methods for determining the thresholds include: threshold estimation based on sample statistics, which extracts feature distributions from known protected building samples to determine statistical thresholds; threshold setting based on expert knowledge, which is determined by cultural relic protection experts based on experience; and threshold optimization based on machine learning, which automatically learns the optimal threshold through training data.

[0047] The initial screening data set refers to the candidate set of buildings initially identified as traditional buildings after morphological-color joint screening. The goal of constructing this set is to cover as many protected buildings as possible, i.e., high recall, while allowing for a certain false positive rate, i.e., low precision, because subsequent value element identification steps will further refine the initial screening results.

[0048] Architectural partial image data refers to high-resolution image data extracted from registered multi-source data, focusing on the exterior details of a single building. Extraction of partial image data requires cropping a local image block centered on the building from remote sensing and ground imagery, based on the building's location information within the spatial reference frame data. Partial image data includes at least building facade images and images of architectural details. The facade images showcase the building's front, side, and rear appearance, while the architectural details images display details such as doors, windows, carvings, and decorations.

[0049] Fine-grained visual recognition models are deep learning models capable of identifying and locating subtle local features in images, such as traditional decorative components, historical traces, and material textures. Unlike conventional object detection models, such as YOLO and Faster R-CNN, which focus on the overall target, fine-grained visual recognition models focus on subtle differences and local features within the target, making them suitable for the refined identification of valuable elements in protected architecture.

[0050] The value element detection results refer to the structured results obtained after detecting the value elements of each initially screened building. These include: indicators of the presence of traditional decorative components, such as the presence of brackets, corbels, and plinths; indicators of the presence of historical traces, such as the presence of weathering marks, historical inscriptions, and repair overlays; and indicators of the presence of traditional material textures, such as the presence of blue brick textures, wood textures, and stone surfaces. When at least one value element is detected, the building is determined to have conservation value and is included in the data of protected buildings.

[0051] In a preferred embodiment, to improve the accuracy and adaptability of the initial screening, the morphological feature parameters and the color feature parameters are jointly screened according to a preset protected building feature threshold to obtain a set of initial screened building data. This can be achieved using a classification method based on a two-dimensional joint screening decision space, specifically including: obtaining the roof slope value and roof outline geometric features from the morphological feature parameters, and the roof hue feature value and roof saturation feature value from the color feature parameters; and determining the roof morphological similarity index for each building based on the roof slope value and the roof outline geometric features, wherein the roof morphological similarity index is the deviation of the roof slope value from a preset slope threshold and the matching of the roof outline geometric features with a preset outline template. The system employs a weighted combination of the roof hue feature value and the roof saturation feature value to determine the roof color deviation index for each building. The roof color deviation index is a weighted combination of the deviation of the roof hue feature value from a preset traditional hue range and the deviation of the roof saturation feature value from a preset saturation range. A two-dimensional joint screening decision space is constructed based on the roof morphology similarity index and the roof color deviation index. In this two-dimensional joint screening decision space, a support vector machine-based classification boundary determination method is used to determine the optimal classification hyperplane for traditional building clusters and modern building clusters. Based on the optimal classification hyperplane, buildings located on one side of the traditional building cluster are identified as the initial screening building set data.

[0052] It should be noted that the two-dimensional joint screening decision space is a two-dimensional feature space with the roof shape similarity index as the horizontal axis and the roof color deviation index as the vertical axis. Each building corresponds to a data point in this space. Support Vector Machine (SVM) is a classification algorithm based on the principle of minimizing structural risk. It separates data points of different categories by finding the optimal classification hyperplane. The optimal classification hyperplane is a linear or nonlinear decision boundary that maximizes the distance between the two classes of data points. Traditional building clusters refer to the aggregation regions of building data points with similar roof shapes and color characteristics in the two-dimensional joint screening decision space. Modern building clusters refer to the aggregation regions of building data points with modern features such as flat roofs and corrugated steel sheets.

[0053] Understandably, the roof features of buildings in traditional villages vary regionally. For example, the gable roofs of Lingnan, the horse-head walls of Hui-style architecture, and the gable roofs of northern courtyard houses all have different slope thresholds and color ranges. Using a fixed threshold for initial screening is difficult to adapt to regional diversity. However, the optimal classification hyperplane based on SVM can adaptively determine the classification boundary according to the distribution of architectural features in a specific village, significantly improving the generalization ability of the initial screening.

[0054] In actual implementation, the roof shape similarity index of each building is first calculated:

[0055] in, This is the roof shape similarity index. To measure the roof slope value, To preset the traditional slope reference value, σ θ For the allowable deviation of slope, The degree of matching between the roof outline and the preset outline template is determined using shape context descriptors. To achieve the maximum matching degree, and The weighting coefficients are such that the sum of the two is 1.

[0056] Calculate the roof color deviation index:

[0057] in, H represents the roof color deviation index, where H is the measured hue value. This is a traditional color tone reference value. Where S represents the permissible range of hue, and S represents the measured saturation value. This is a traditional saturation reference value. Within the allowable range of saturation, and These are the weighting coefficients.

[0058] Construct a two-dimensional joint screening decision space, in order to The horizontal axis is... Using the vertical axis as the ordinate, data points for all buildings within the village are plotted in this space. A support vector machine based on the radial basis function kernel (RBF kernel) is used for training to determine the optimal classification hyperplane.

[0059] in, For feature vectors, For Lagrange multipliers, These are category labels; +1 indicates traditional architecture, and -1 indicates modern architecture. Here, is the RBF kernel function, b is the bias term, and γ is the kernel parameter.

[0060] The optimal hyperplane parameters are determined through cross-validation. Buildings with a value greater than 0 are identified as initial screening building sets.

[0061] In a preferred embodiment, to improve the accuracy and comprehensiveness of value element detection, the step of detecting value elements based on the local building image data using a fine-grained visual recognition model to obtain value element detection result data can employ a multi-level feature fusion detection strategy. Specifically, this includes: inputting the local building image data into a pre-trained multi-level feature pyramid network to extract multi-scale feature maps from the local building image data, wherein the multi-scale feature maps include shallow texture feature maps, mid-level structural feature maps, and deep semantic feature maps; based on the shallow texture feature maps, using a texture analysis algorithm based on gray-level co-occurrence matrix to detect traditional material texture features, obtaining material texture detection results; based on the mid-level structural feature maps, using a component localization network based on deformable convolution to detect the spatial position and geometric shape of traditional decorative components, obtaining decorative component detection results; based on the deep semantic feature maps, using a historical trace recognition network based on an attention mechanism to detect wall weathering traces, historical inscriptions, and repair overlay traces, obtaining historical trace detection results; and fusing the material texture detection results, the decorative component detection results, and the historical trace detection results to obtain value element detection result data.

[0062] It should be noted that the Multi-Level Feature Pyramid Network (FPN) is a convolutional neural network architecture that constructs multi-scale feature representations through top-down paths and lateral connections, enabling the extraction of rich semantic information at different resolution levels. Shallow texture feature maps refer to high-resolution feature maps output from the shallow layers of the network, preserving rich texture details and suitable for material texture detection. Mid-layer structural feature maps refer to medium-resolution feature maps output from the middle layers of the network, taking into account both spatial location and semantic information, suitable for component localization. Deep semantic feature maps refer to low-resolution feature maps output from the deep layers of the network, possessing a large receptive field and high-level semantic abstraction capabilities, suitable for historical trace recognition. Gray-level co-occurrence matrix (GLCM) is a second-order statistical method describing the statistical properties of image texture. By calculating the gray-level co-occurrence probability distribution of pixel pairs at different directions and distances, it extracts texture features such as contrast, correlation, energy, and uniformity. Deformable convolution is a convolution operation that adaptively adjusts the sampling position of the standard convolution kernel by learning offsets, better adapting to targets with large geometric deformations. Attention mechanisms are mechanisms that enhance key information and suppress irrelevant information by calculating importance weights for feature channels or spatial locations.

[0063] Understandably, the value elements of traditional village architecture are multi-scale and multi-type: material textures, such as the patterns on blue bricks and the texture of wooden structures, belong to micro-textural features and need to be detected on shallow, high-resolution feature maps; decorative components, such as brackets, corbels, and plinths, belong to meso-structural features and need to be located on mid-level feature maps; historical traces, such as weathering, historical inscriptions, and layers of repair, belong to macro-semantic features and need to be identified on deep feature maps. Single-scale detection methods cannot simultaneously cover these three types of value elements, while multi-level feature fusion detection strategies, by deploying specialized detection algorithms at different levels, achieve comprehensive and accurate identification of value elements.

[0064] In practical implementation, the multi-level feature pyramid network uses ResNet-50 as the backbone network. Through top-down upsampling and lateral connections, it constructs four feature pyramids from P2 to P5. At the P2 level, with the highest resolution, a GLCM-based texture analysis algorithm is used for material texture detection. The P2 feature map is converted to a grayscale image, and the gray-level co-occurrence matrices in four directions (0°, 45°, 90°, and 135°) are calculated. Contrast, correlation, energy, and uniformity are extracted to construct a texture feature vector. A trained support vector machine classifier then classifies the texture feature vector into traditional material textures, such as brick, wood, and stone, or modern material textures, such as ceramic tiles, paint, and metal plates, thus obtaining the material texture detection results.

[0065] At the P3 and P4 levels, a deformable convolution-based component localization network is used for decorative component detection. The deformable convolution learns a two-dimensional offset at each sampling point of a standard 3×3 convolution kernel, enabling the kernel to adaptively fit the irregular shape of the decorative component. Let the sampling position of the standard convolution kernel be... The output of deformable convolution is:

[0066] in, For output position, For preset offset, To learn the position offset, For convolution weights, For input features.

[0067] The feature map enhanced by deformable convolution is input into the Region Proposal Network (RPN) to generate candidate boxes for decorative components. Then, the region of interest alignment and fully connected layers are used for classification and regression to obtain the detection results of decorative components.

[0068] At the P5 level, a historical trace recognition network based on an attention mechanism is employed. First, a channel attention module weights the features from different channels. Then, a spatial attention module weights the spatial location of the feature maps, highlighting areas with historical traces. Finally, a fully connected layer performs multi-label classification, such as weathering traces, inscriptions, and repair / overlay traces, to obtain the historical trace detection results.

[0069] By integrating the detection results from the three levels and employing a confidence-weighted voting strategy, the final value element detection result data is obtained. When at least one value element is detected, the building is determined to be a protected building, and its value element type, quantity, and spatial location are recorded to construct protected building object data.

[0070] Step S30: Integrate the protected building object data, existing protected object data, and natural geographic element data to construct a spatial correlation map and perform protection priority learning and zoning boundary generation to obtain protected zoning vector data.

[0071] It should be noted that existing protected object data refers to the list of protected objects already published by relevant departments or cultural heritage institutions prior to this application, including at least spatial data of immovable cultural relics and spatial data of historical environmental elements. Spatial data of immovable cultural relics refers to the spatial location information of fixed buildings or sites with historical, artistic, and scientific value. Spatial data of historical environmental elements refers to elements related to the historical environment of the protected buildings, such as ancient trees, ancient wells, and ancient bridges. Natural geographical element data refers to naturally formed topographic and geomorphic elements within the village, including at least road data, water system data, and topographic data. A spatial relationship diagram is a graph-structured data model with spatial elements as nodes and spatial relationships as edges, used to describe the spatial relationships between protected elements. Protected area vector data refers to vector data containing the boundaries of the core protected area and the construction control zone.

[0072] Understandably, the delineation of traditional village protection zones has long relied on the subjective experience of planners, lacking objective and quantitative scientific basis. Different planners may produce significantly different zoning results for the same village, and the zoning boundaries are often presented as simple geometric shapes, such as circular buffer zones, which are disconnected from natural geographical boundaries. This implementation method constructs a spatial relationship graph, uniformly modeling elements such as protected buildings, cultural relics, ancient trees, roads, and water systems as graph nodes, and modeling relationships such as spatial adjacency, visual corridors, and historical axes as graph edges. It employs a graph neural network to learn the protection priority of each element, achieving data-driven and objectified zoning generation.

[0073] In one feasible implementation, step S30 may include: acquiring existing protected object data and natural geographic element data, wherein the existing protected object data includes at least immovable cultural relic spatial data and historical environmental element spatial data, and the natural geographic element data includes at least road data, water system data, and topographic data; fusing the protected building object data, the existing protected object data, and the natural geographic element data to construct a multi-source spatial dataset; based on the multi-source spatial dataset, extracting the spatial adjacency relationships and topological connectivity relationships of each element node to construct a spatial association graph; based on the spatial association graph, using a graph neural network to perform node protection priority embedding learning to obtain the protection priority vector of each node; calculating the spatial distribution density of protected elements according to the protection priority vector of each node to obtain protected element density field data; and generating zoning boundaries based on the protected element density field data and the natural geographic element data to obtain protected zoning vector data.

[0074] It should be noted that road data refers to the spatial data of the village's transportation network, including streets, alleys, walkways, and ancient paths, encompassing information such as road centerlines, road widths, and road paving types. Water system data refers to the spatial data of water features within the village, such as rivers, streams, ponds, and wells, including information such as water system centerlines, water boundaries, and water types. Topographic data describes the undulating terrain of the village, typically expressed as a Digital Elevation Model (DEM) or contour lines, including information such as elevation points, contour lines, slope, and aspect.

[0075] Understandably, the formation and development of traditional villages are deeply influenced by their natural geographical environment. "Located near mountains and water," "adapting to the terrain," and "choosing to live near water" are common principles in the site selection and layout of traditional villages. Roads form the framework of the village's internal spatial organization, water systems are essential for production and daily life, and topography determines the overall spatial pattern of the village. These natural geographical elements are not only components of the village's spatial form but also important bases for delineating protected area boundaries. That is, the boundaries should conform as closely as possible to the natural geographical boundaries, meeting both protection needs and facilitating management implementation.

[0076] A multi-source spatial dataset refers to a unified spatial database formed by fusing data on protected buildings, existing protected objects, and natural geographic features. Data fusion includes unifying spatial location (converting all data to the same coordinate reference system), unifying attribute structure (standardizing field naming, data types, and coding standards), and establishing semantic relationships (establishing spatial and logical relationships between different features).

[0077] Spatial adjacency refers to the direct proximity or adjacency of two spatial elements in geographic space, typically determined through distance thresholds or topological analysis. Two elements are considered to have a spatial adjacency relationship when the Euclidean distance between them is less than a preset threshold. Spatial adjacency reflects the physical proximity and mutual influence between elements, such as the stylistic harmony between adjacent buildings or the landscape association between buildings and ancient trees.

[0078] Topological connectivity refers to the logical connection or association between two spatial elements, going beyond simple geometric proximity. Topological connectivity includes: visual corridor relationships, meaning unobstructed visual connection between two elements; and historical axis relationships, meaning two elements are separated along historical streets or ceremonial axes. Topological connectivity reflects the cultural relevance and spatial organization logic between elements, and is key to understanding the spatial pattern of traditional villages.

[0079] A spatial association graph is a graph-structured data model that uses spatial elements as nodes and spatial relationships as edges to describe the spatial relationships between protected elements. The formal definition of a spatial association graph is G=(V,E), where V is the set of nodes, each corresponding to a spatial element; and E is the set of edges, each corresponding to a spatial relationship between two nodes. A spatial association graph can be a homogeneous graph, meaning all nodes and edges are of the same type, or a heterogeneous graph, meaning nodes and edges have multiple types. This implementation uses a heterogeneous graph to fully express the diverse element types and relationship types in traditional villages.

[0080] A protection priority vector is a low-dimensional vector representation of the importance of protection for each node, learned through a graph neural network. Each dimension of the protection priority vector corresponds to a dimension of protection value, such as historical value, artistic value, scientific value, and social value. The magnitude of the vector reflects the overall degree of protection importance, while the direction of the vector reflects the constituent characteristics of the protection value. Using the protection priority vector, all protection elements in a village can be quantitatively ranked, providing a scientific basis for zoning generation.

[0081] Density field data of protected elements refers to continuous field data describing the spatial density of protected elements. A density field is a method that transforms discrete spatial elements into a continuous spatial distribution representation. Through kernel density estimation or other spatial interpolation methods, the distribution density value of protected elements is calculated at each spatial location. High-value regions of the density field correspond to the core areas of densely distributed protected elements, while low-value regions correspond to the peripheral areas of sparsely distributed protected elements. The gradient of the density field reflects the spatial variation trend of the protected element distribution.

[0082] Protected area vector data refers to vector data that includes the boundaries of the core protected area and the construction control zone. The core protected area refers to the area within a traditional village where historical buildings and buildings with traditional features are concentrated and well-preserved in terms of spatial pattern and appearance. This area is the most important part of the protection work and is usually subject to the strictest construction controls. The construction control zone refers to the area outside the core protected area that needs to be controlled in order to protect the overall appearance and natural environment of the village. It is usually subject to moderate construction controls and allows new construction and renovation activities that meet the appearance requirements.

[0083] In a preferred embodiment, to fully consider the unique spatial relationship types in traditional villages, the step of using a graph neural network to perform node protection priority embedding learning based on the spatial relationship graph to obtain the protection priority vector of each node can employ a heterogeneous graph attention network. Specifically, this includes: determining the node attribute features of each node in the spatial relationship graph, wherein the node attribute features include building age attributes, building structure type attributes, historical event association attributes, and landscape integrity attributes; and determining the edge type features and edge weight features of each edge in the spatial relationship graph, wherein the edge type features include spatial adjacency edges and line-of-sight edges. For corridor edges and historical axis edges, the edge weight features are calculated based on spatial distance, historical association strength, and visual connectivity. Heterogeneous graph neural network input data is constructed based on the node attribute features, edge type features, and edge weight features. This input data is then fed into a pre-defined heterogeneous graph attention network, which comprises multiple heterogeneous graph attention layers. Each layer employs a differentiated attention coefficient calculation strategy for different types of edges. Node features are aggregated and propagated through these multiple heterogeneous graph attention layers to obtain the protection priority embedding vector for each node. Based on this protection priority embedding vector, a softmax-normalized protection level classification method is used to determine the protection priority vector for each node.

[0084] It should be noted that the Heterogeneous Graph Attention Network (HAN) is a graph neural network architecture capable of handling various node and edge types in heterogeneous graphs. By introducing a type-specific attention mechanism, it achieves effective aggregation of heterogeneous information. Spatial adjacency edges refer to edges where two spatial elements are directly adjacent in geographic space. Visual corridor edges refer to edges where two spatial elements have unobstructed visual connectivity; in traditional villages, visual corridors are crucial for preserving the overall landscape and layout of the village. Historical axis edges refer to the sequential relationships between elements distributed along historical streets or ceremonial axes, reflecting the historical spatial organization of the village.

[0085] Understandably, this implementation method, by introducing local-specific edge types such as historical axis edges, enables graph neural networks to learn cultural spatial relationships that go beyond pure geometric space, significantly improving the accuracy and cultural rationality of protection priority learning.

[0086] In practice, the first step is to construct a multi-source spatial dataset. Each protected building in the protected building object data is treated as a node, with node attributes including: building age; building structure type, such as wood, brick-wood, or stone; historical event associations, such as whether any major historical events occurred there or whether there are records of famous people residing there; and architectural integrity, such as the proportion of original components preserved and whether it has undergone improper repairs. Each protected object in the immovable cultural relic spatial data and the historical environmental element spatial data is also treated as a node and assigned corresponding node attributes.

[0087] Extract spatial relationships to construct an edge set. Spatial adjacency edges: When the Euclidean distance between two nodes is less than a preset threshold, such as 50 meters, a spatial adjacency edge is established, with the edge weight being the reciprocal of the distance. Visual corridor edges: Using a visual analysis method based on a Digital Elevation Model (DEM), it is determined whether there is an unobstructed visual corridor between two nodes. If so, a visual corridor edge is established, with the edge weight being a function of the width and length of the visual corridor. Historical axis edges: Based on village historical documents and field surveys, the main historical streets and ceremonial axes are identified. Nodes distributed along the axes are sequentially connected to form historical axis edges, with the edge weight being the weight of the node's sequential position on the axis. Construct a heterogeneous graph attention network. Let the node type set be T = {buildings, cultural relics, ancient trees, roads, water systems, mountains}, and the edge type set be E = {spatial adjacency, visual corridor, historical axis}. For each edge type Φ∈E, type-specific attention coefficients are defined, and attention weights are obtained through softmax normalization. Node features are then aggregated based on these attention weights. The aggregated features from different edge types are fused using a semantic attention mechanism. After passing through multiple heterogeneous graph attention layers, the protection priority embedding vector for each node is obtained. A softmax classifier is then used to classify protection levels, such as Level 1, Level 2, and Level 3 protection, resulting in the protection priority vector.

[0088] In a preferred embodiment, to ensure that the zoning boundary conforms to natural geographical features and adapts to the spatial pattern of traditional villages, the generation of zoning boundary based on the protected element density field data and the natural geographical element data to obtain protected zoning vector data can be achieved using a boundary generation method based on level set morphological evolution. Specifically, this includes: using an adaptive bandwidth kernel density estimation method based on the protected element density field data to determine the peak region and gradient abrupt change region of the spatial distribution of protected elements; and extracting natural boundary constraint lines based on the water system data and topographic data in the natural geographical element data, wherein the natural boundary constraint lines include the center line of rivers, the shoreline of ponds, and the ridgeline. The initial seed region of the core protection area is determined based on the peak region, and the initial boundary of the transition zone between the core protection area and the construction control zone is determined based on the gradient abrupt change region. A morphological evolution algorithm based on the level set method is used, with the natural boundary constraint line as the hard constraint boundary and the initial boundary of the transition zone as the initial zero level set, to perform zoning boundary evolution calculation. During the zoning boundary evolution calculation, road data from the natural geographic element data is introduced as a permeation channel constraint, so that the evolved zoning boundary forms a natural transition along the road direction. When the zoning boundary evolution calculation reaches the convergence condition, the zero level set curve is extracted as the final zoning boundary line, and vector data of the core protection area and the construction control zone are generated based on the zoning boundary line.

[0089] It should be noted that adaptive bandwidth kernel density estimation is a density estimation method that adaptively adjusts the kernel function bandwidth based on the local density of sample points. A smaller bandwidth is used in densely populated areas of protected elements to improve resolution, while a larger bandwidth is used in sparse areas to ensure smoothness. Peak regions refer to the local maxima of the protected element density field, corresponding to the core region with the highest protection value. Regions with steep gradient changes refer to areas where the gradient of the protected element density field changes drastically, corresponding to the transition zone between the core and peripheral regions. The level set method is a mathematical method that implicitly represents a closed curve as a higher-dimensional function zero-level set, achieving topological changes and smooth deformation of the curve by evolving this higher-dimensional function. Hard constraint boundaries refer to fixed boundaries that are not allowed to be crossed during the evolution process. Penetration channel constraints allow for local adjustments to zoning boundaries along specific linear elements, such as roads, forming a natural transition that meets actual management needs.

[0090] Understandably, traditional villages are often built along waterways and take advantage of the terrain, with natural geographical elements such as rivers, ponds, and ridges forming the natural spatial boundaries of the villages. Meanwhile, streets and roads are a crucial framework for the internal spatial organization of the villages, and the transition of zoning boundaries along these roads not only facilitates management but also respects the historical spatial pattern of the villages. Existing zoning methods often employ simple geometric buffer zones, which are disconnected from natural geographical boundaries and historical spatial patterns. This application, however, through the morphological evolution of level sets, uses natural boundaries as hard constraints and roads as permeation channels, generating zoning boundaries that both conform to natural geographical features and historical spatial logic.

[0091] In actual implementation, adaptive bandwidth kernel density estimation is performed first. Let the set of protected element nodes be... ,in Let be the plane coordinates of the nodes. The kernel density estimate is:

[0092] in, The adaptive bandwidth is determined based on the local density of node i:

[0093] in, As the global baseline bandwidth, Let K be the average bandwidth and K be the kernel function for kernel density estimation.

[0094] The peak region is determined by finding local maxima in the density field, and the density gradient is calculated. The magnitude of the gradient determines the region of steep gradient change, that is, the region where the gradient magnitude is greater than a preset threshold.

[0095] Constructing level set functions The function is initialized with a signed distance function to determine the evolution equation of the level set. The evolution process is smoothed by a curvature term, while natural boundary constraint terms and infiltration channel constraint terms are introduced to adjust the evolution speed. When the iterative change of the level set function is less than the preset convergence threshold, the evolution stops, and the current zero level set is extracted as the final boundary between the core protection area and the construction control zone. Subsequently, vector polygons are extracted according to the final boundary to obtain vector data of the core protection area and vector data of the construction control zone, thus completing the generation of protection zoning vector data.

[0096] Step S40: Construct a digital twin baseline model based on the protected building object data, wherein the digital twin baseline model includes a multi-level semantic model of the building appearance and a three-dimensional geometric benchmark model.

[0097] It should be noted that the digital twin baseline model refers to a high-precision 3D model that accurately maps the appearance of a protected building in a digital way, serving as a benchmark for subsequent comparative detection using surveillance videos. The multi-level semantic model of the building appearance refers to a structured model that decomposes the building appearance into multiple semantic levels, such as roof, doors and windows, exterior walls, and decorative components, and establishes topological relationships between these levels. The 3D geometric benchmark model refers to a 3D model containing precise 3D geometric and texture information of the building, used for cross-modal registration with surveillance videos.

[0098] Understandably, traditional building exterior inspection methods lack precise benchmarks, typically relying on historical photographs or human memory, resulting in high subjectivity and low accuracy. This application addresses this by constructing a digital twin baseline model, solidifying the current state of the building exterior into a millimeter-level precision 3D model, providing an objective and quantifiable benchmark for subsequent anomaly detection.

[0099] In one feasible implementation, step S40 may include: extracting architectural attribute information and spatial location information of each protected building from the protected building object data; performing three-dimensional geometric reconstruction based on the architectural attribute information and the spatial location information to obtain three-dimensional geometric model data; performing appearance texture mapping on the three-dimensional geometric model data to obtain three-dimensional texture model data; constructing a multi-level semantic model of building appearance based on the three-dimensional texture model data, wherein the multi-level semantic model of building appearance includes at least a roof semantic layer, a door and window semantic layer, an exterior wall semantic layer, and a decorative component semantic layer; and associating and binding the three-dimensional texture model data with the multi-level semantic model of building appearance to obtain a digital twin baseline model.

[0100] It should be noted that building attribute information refers to attribute data that describes the basic characteristics and status of a protected building, including at least the building name, building age, building structure type, number of floors, building area, building height, protection level, historical background, and current status description.

[0101] Spatial location information refers to the geographical location information of the protected building in a unified spatial coordinate reference system, including at least the planar coordinates of the building's center point, such as longitude, latitude or projected coordinates, building footprint, building outline polygon, building height, etc.

[0102] 3D geometric reconstruction refers to the process of extracting the 3D geometric information of a building from multi-source spatial data and constructing a 3D model with accurate geometric form. Methods for 3D geometric reconstruction include image-based reconstruction, such as Structure of Motion (SfM) and Multi-View Stereo (MVS); point cloud-based reconstruction, such as laser scanning point cloud processing; and CAD-based modeling, such as manual modeling and parametric modeling. This implementation adopts a 3D reconstruction method based on multi-view imagery because traditional village buildings have complex and detailed appearances, and image data can fully capture their appearance features, with relatively low acquisition costs.

[0103] Texture mapping refers to the process of mapping the texture information of a two-dimensional image onto the surface of a three-dimensional geometric model. Texture mapping gives the 3D model realistic appearance, color, and material texture, not only improving the visualization effect but, more importantly, providing comparable baseline texture information for subsequent cross-modal registration and anomaly detection. The quality of texture mapping directly affects the accuracy of subsequent anomaly detection, requiring precise correspondence between texture and geometry, seamless handling of texture seams, and appropriate selection of texture resolution.

[0104] A multi-level semantic model of building appearance refers to a model that decomposes the building appearance into multiple semantic levels and performs structured modeling of each level. Semantic levels refer to the classification hierarchy based on the function, location, and morphological characteristics of building components, and include at least: roof semantic level, door and window semantic level, exterior wall semantic level, and decorative component semantic level. The roof semantic level includes roof forms, such as gable roof, hip roof, hipped roof, and pyramidal roof; roof materials, such as blue tiles, glazed tiles, and thatch; ridge decorations, such as roof ornaments and finials; and eaves structures, such as flying eaves and brackets. The door and window semantic level includes door forms, such as panel doors, lattice doors, and hanging flower doors; window forms, such as straight-lattice windows, threshold windows, and hinged windows; and door and window decorations, such as door studs, door nails, and window lattice patterns. The semantic layer of exterior walls includes wall materials, such as blue bricks, red bricks, stone, and adobe; wall structures, such as hollow brick walls, solid masonry walls, and framed walls; and wall decorations, such as brick carvings, stone carvings, plasterwork, and painted decorations. The semantic layer of decorative components includes eaves decorations, such as brackets, corbels, and hanging ornaments; roof ridge decorations, such as roof ornaments, finials, and patterned bricks; wall decorations, such as plinths, screen walls, and gate towers; and other detailed decorations, such as column bases, railings, and steps.

[0105] Topological relationships between semantic levels refer to the spatial and logical relationships between different semantic levels, including: adjacency, subordination, and hierarchy. Adjacency means two semantic levels are directly adjacent in space, such as a roof and an exterior wall, or doors and windows and an exterior wall. Subordination means one semantic level is a component of another, such as doors and windows belonging to an exterior wall, or brackets belonging to eaves. Hierarchical relationships refer to a hierarchical inclusion relationship between different semantic levels, such as decorative components belonging to a roof or exterior wall, forming a hierarchical structure of building → exterior wall → doors and windows → window frames.

[0106] A digital twin baseline model is a high-precision 3D model that accurately maps the current state of a protected building's exterior in a digital format, serving as a benchmark for subsequent comparative detection using surveillance videos. The digital twin baseline model contains not only 3D geometric and textural information, but also multi-level semantic and topological relational information, enabling subsequent anomaly detection to perform refined analysis at the semantic level, rather than relying solely on fuzzy judgments based on pixel changes.

[0107] In a preferred embodiment, to improve the accuracy and structural rationality of 3D geometric reconstruction, the 3D geometric reconstruction based on the building attribute information and the spatial location information to obtain 3D geometric model data can employ a reconstruction method with structural constraint optimization. Specifically, this includes: extracting multi-view image data corresponding to each protected building from the registered multi-source data based on the spatial location information; generating initial sparse point cloud data for each protected building using a motion-reconstruction-based 3D sparse point cloud reconstruction algorithm based on the multi-view image data; obtaining dense point cloud data for each protected building using a multi-view stereo-based dense point cloud generation algorithm based on the initial sparse point cloud data; obtaining initial 3D mesh model data for each protected building using a Poisson surface reconstruction-based triangular mesh generation algorithm based on the dense point cloud data; and optimizing the initial 3D mesh model data based on the building structure type attribute in the building attribute information, wherein the structural constraint optimization includes roof surface smoothness constraints, wall verticality constraints, and component symmetry constraints, to obtain 3D geometric model data.

[0108] It should be noted that motion reconstruction is a computer vision method that analyzes the correspondence of feature points in multi-view images while simultaneously estimating camera pose and 3D point cloud structure. Multi-view stereo is a method that generates high-density point clouds through dense matching based on sparse point clouds. Poisson surface reconstruction is a surface reconstruction method that generates smooth triangular meshes from directed point clouds by solving the Poisson equation. Roof surface smoothness constraint refers to applying curvature continuity constraints to the roof surface to ensure that the reconstructed roof surface is smooth and natural. Wall verticality constraint refers to applying geometric constraints perpendicular to the ground to the wall plane to ensure that the wall is not tilted. Component symmetry constraint refers to applying symmetry constraints to symmetrical building components, such as doors, windows, and brackets, to ensure the geometric symmetry of the reconstructed components.

[0109] Understandably, traditional village buildings are mostly timber-framed or brick-and-timber structures with a clear structural logic: roofs are curved or sloping, walls are vertical planes, and components such as doors and windows often have a symmetrical layout. Purely point cloud-based surface reconstruction methods may result in non-physical phenomena such as uneven roofs, tilted walls, and asymmetrical components due to point cloud noise or occlusion. This implementation method introduces structural constraint optimization driven by architectural structural knowledge, ensuring that the reconstruction results not only conform to visual observation but also to the physical logic of the building structure, significantly improving the accuracy and usability of the reconstruction model.

[0110] In actual implementation, the camera pose and sparse point cloud are first recovered from multi-view images using a motion reconstruction algorithm. Let the projection matrix of the i-th camera be... ,in This is the intrinsic parameter matrix. Let be a rotation matrix. This is the translation vector. The reprojection error is minimized through feature point matching and bundling adjustments.

[0111] in, Let J be the observed pixel coordinates of the j-th 3D point in the i-th camera. Let j be the spatial coordinates of the j-th 3D point. This is the projection function.

[0112] Based on sparse point clouds, a multi-view stereo algorithm is used for dense matching to generate dense point clouds. By estimating the normal vector and confidence level of each point, a directed point cloud is constructed.

[0113] An initial triangular mesh is generated from the directed point cloud using Poisson surface reconstruction. Structural constraint optimization is performed based on building structure type attributes. Let the initial mesh model be... ,in For the set of vertices, Let the set be a set of facets. Define the structural constraint energy function. :

[0114] Roof surface smooth confinement energy :

[0115] in, For the set of vertices of the roof area, Let v be the average curvature at vertex v.

[0116] Wall verticality constraint energy :

[0117] in, For the set of vertices of the wall region, Let v be the normal vector at vertex v. This is the ground normal vector.

[0118] Component symmetry constraint energy :

[0119] in, It is a set of symmetric vertex pairs. It is a symmetry transformation matrix, determined based on the component's axis of symmetry or plane of symmetry.

[0120] By minimizing the total energy function The initial mesh is optimized by structural constraints to obtain three-dimensional geometric model data that conforms to the logic of building structure.

[0121] In a preferred embodiment, to achieve refined semantic modeling of the building's appearance, the construction of a multi-level semantic model of the building's appearance based on the three-dimensional texture model data can employ a multi-level segmentation method based on geometric analysis and texture features. Specifically, this includes: based on the three-dimensional texture model data, using a region-growing-based roof patch segmentation algorithm to extract a set of roof geometric patches, obtaining a roof semantic layer; based on the three-dimensional texture model data, using a template-matching-based door and window component detection algorithm to extract a set of door and window geometries, wherein the set of door and window geometries includes door frame outlines, window frame outlines, and window lattice patterns, obtaining a door and window semantic layer; based on the three-dimensional texture model data, using a normal-based... A mass clustering algorithm for wall region segmentation extracts a set of geometric facets of the exterior wall and classifies and labels these facets according to their material texture features, resulting in a semantic layer for the exterior wall. Based on the 3D texture model data, a scale-invariant feature transformation-based algorithm for locating decorative components is used to extract a set of geometric shapes for decorative components, including eaves, brackets, plinths, and screen walls, resulting in a semantic layer for decorative components. Topological relationships are established between the semantic layers for the roof, doors and windows, exterior walls, and decorative components, including adjacency, dependency, and hierarchical relationships, resulting in a multi-level semantic model of the building's exterior.

[0122] It should be noted that region growing is a segmentation method that starts from a seed point and gradually expands the region according to a preset growth criterion. Template matching is a method for object detection that calculates the similarity between a template image and a target image. Normal vector clustering is a method for grouping 3D point clouds or patches based on the direction of surface normal vectors. Scale-invariant feature transformation is a local feature descriptor that is invariant to changes in scale, rotation, and illumination. Adjacency relationship refers to the relationship where two semantic layers are directly adjacent in space. Subordination relationship refers to the relationship where one semantic layer is a component of another semantic layer, such as doors and windows belonging to the exterior wall. Hierarchical relationship refers to the hierarchical inclusion relationship between different semantic layers, such as decorative components belonging to the roof or exterior wall.

[0123] Understandably, a multi-level semantic model of a building's exterior is not merely a simple geometric division, but also requires establishing topological relationships between levels. For example, doors and windows belong to the exterior walls, and brackets belong to the eaves. This hierarchical relationship is crucial for subsequent anomaly detection. When an anomaly is detected in an exterior wall area, it is necessary to simultaneously check whether the doors and windows belonging to that exterior wall are also affected. This application, by explicitly modeling the topological relationships between semantic levels, makes the digital twin baseline model not only a static three-dimensional model, but also an intelligent benchmark with structural knowledge.

[0124] In practice, roof panel segmentation employs a region growing algorithm based on normal vector consistency. Seed points are selected within the roof region, i.e., points whose normal vectors are close to vertically upward. The growth criterion is that the angle between the normal vectors of adjacent panels is less than a threshold. For example, 15°, and the distance from the patch to the fitted plane is less than the threshold. For example, 0.1 meters. After growth is complete, the extracted roof patch set is subjected to planar fitting and surface parameterization to obtain the roof semantic layer.

[0125] The detection of door and window components adopts a template matching method based on Hu rectangular descriptors. A template library of traditional door and window components is pre-established, including door frame templates, window frame templates, and common window lattice pattern templates. The template is slidably searched in the 3D texture model to determine the Hu moment similarity between the template and the local area. When the Hu moment similarity is less than a threshold, the door and window component is detected, and its geometric position and type are recorded to obtain the semantic layer of the door and window.

[0126] Wall area segmentation employs a K-means clustering method based on normal vectors. The normal vectors of all facets are used as feature vectors and clustered into K categories, with facets whose normal vectors are nearly horizontal classified as exterior walls. External wall facets are then classified into material types based on texture features extracted through GLCM, such as blue brick, red brick, stone, and plaster, resulting in the exterior wall semantic layer.

[0127] The localization of decorative components employs a feature matching method based on scale-invariant feature transformation. A feature template library for decorative components, such as eaves, brackets, plinths, and screen walls, is pre-established. Multi-scale feature matching is performed in a 3D texture model. After detecting a decorative component, its type and spatial location are recorded to obtain the semantic layer of the decorative component.

[0128] Establish topological relationships. Adjacency relationships are determined by whether two semantic layers share a boundary or have a minimum distance less than a threshold. Subordination relationships are predefined based on architectural structural knowledge, such as doors and windows belonging to exterior walls, and brackets belonging to eaves. Hierarchical relationships are defined according to component dimensions, such as roofs and exterior walls being the first level, doors and windows the second level, and decorative components the third level. All topological relationships are stored through a graph structure to obtain a multi-level semantic model of the building's appearance.

[0129] By associating and binding 3D texture model data with multi-level semantic models of building appearance, texture coordinate mapping and geometric index corresponding to each semantic level are established to obtain a complete digital twin baseline model.

[0130] Step S50: Acquire monitoring video stream data, and perform cross-modal registration of the monitoring video stream data with the digital twin baseline model to obtain registered monitoring data.

[0131] It should be noted that surveillance video stream data refers to continuous video data collected in real time by surveillance cameras within the village, including timestamp information and spatial location information of the collecting devices. Cross-modal registration refers to the process of spatially aligning two-dimensional video images with a three-dimensional digital twin baseline model, establishing the correspondence between video pixels and surface points of the three-dimensional model. Registered monitoring data refers to enhanced data that includes video pixel information, semantic category information, and baseline texture information after cross-modal registration.

[0132] Understandably, surveillance video and 3D models belong to different data modalities—two-dimensional images and three-dimensional geometry—and camera angles, lighting conditions, and weather conditions are constantly changing, making direct comparative analysis extremely difficult. This application utilizes cross-modal registration to project the semantic and texture information of the 3D model into the video image space. This ensures that video frames not only contain the original pixel information but also the semantic category and baseline texture corresponding to each pixel, providing a precise reference system for subsequent anomaly detection.

[0133] In one feasible implementation, step S50 may include: acquiring surveillance video stream data, wherein the surveillance video stream data includes timestamp information and spatial location information of the acquisition device; performing keyframe extraction and quality enhancement processing on the surveillance video stream data to obtain preprocessed video frame sequence data; determining the initial spatial attitude parameters of each video frame based on the spatial location information of the acquisition device and the spatial reference frame data; refining the spatial attitude of the preprocessed video frame sequence data using a feature matching-based cross-modal registration algorithm based on the three-dimensional geometric reference model in the digital twin baseline model to obtain refined video frame attitude parameters; and performing pixel-level spatial alignment between the video frame sequence data and the digital twin baseline model according to the refined video frame attitude parameters to obtain registered monitoring data.

[0134] It should be noted that surveillance video stream data refers to continuous video data collected in real time by surveillance cameras within the village. These cameras are typically fixedly installed in key locations within the village, such as entrances, squares, street intersections, and around important buildings, and have a fixed field of view and monitoring range. Surveillance video stream data not only contains image frame sequences but also metadata information. The timestamp information records the acquisition time of each frame, and the spatial location information of the acquisition device records the camera's installation coordinates and orientation, such as pitch, yaw, and roll angles.

[0135] Keyframe extraction is a method for selecting representative frames from a continuous video stream for subsequent processing. Due to the massive data volume of surveillance video streams—for example, a 1080p resolution, 25fps camera generates approximately 2TB of data daily—processing all frames is neither practical nor necessary. Keyframe extraction analyzes the temporal changes in video content, selecting frames with significant content changes as keyframes, thus reducing data volume while retaining important information. Keyframe extraction strategies include: extraction based on inter-frame differences (extracting keyframes when the pixel difference between frames exceeds a threshold); extraction based on motion analysis (extracting keyframes when significant motion occurs in the frame); and extraction based on fixed intervals (extracting one keyframe every N frames), etc.

[0136] Quality enhancement processing is a process that improves the image quality of video frames. Surveillance videos may suffer from quality issues such as blurriness, noise interference, low contrast, and color distortion due to factors such as camera performance, lighting conditions, and weather conditions. Quality enhancement processing includes: deblurring, which eliminates motion blur or out-of-focus blur using deconvolution or deep learning deblurring algorithms; denoising, which eliminates random noise using temporal or spatial filtering; contrast enhancement, which improves contrast through histogram equalization or adaptive enhancement; and super-resolution reconstruction, which improves image resolution using deep learning algorithms.

[0137] Initial spatial attitude parameters refer to the camera pose parameters of video frames preliminarily determined based on the spatial location information of the acquisition equipment and the spatial reference frame data. Since the installation position of the surveillance camera has been determined through measurements during installation, including GPS coordinates, installation height, and orientation angle, it can be directly used as the initial pose parameters. The initial spatial attitude parameters include the three-dimensional coordinates (X, Y, C, D) of the camera center. c ,Y c Z c The initial spatial attitude parameters, namely pitch angle Φ, yaw angle ω, and roll angle κ, provide good initial values ​​for subsequent refined registration, reducing the search space and improving registration efficiency and success rate.

[0138] The refined video frame pose parameters refer to the accurate camera pose obtained after optimizing and correcting the initial pose parameters through a cross-modal registration algorithm. Due to factors such as installation measurement errors, long-term camera deformation under stress, and wind vibration, the initial pose parameters may contain deviations. Refined registration establishes the feature correspondence between the video frame and the 3D model, solving for the precise pose parameters to ensure accurate pixel-level alignment between the building's appearance in the video frame and the 3D model.

[0139] Pixel-level spatial alignment refers to establishing a precise spatial correspondence between each pixel in a video frame and its corresponding point on the surface of a 3D model. Pixel-level alignment requires not only correct overall pose but also precise correspondence in the projected position of each pixel, which is a prerequisite for achieving refined anomaly detection. The accuracy of pixel-level alignment directly affects the accuracy of subsequent anomaly detection, and typically requires a reprojection error of less than 3 pixels.

[0140] In a preferred embodiment, to address the modal differences between the 3D model and the 2D video, the 3D geometric reference model in the digital twin baseline model is used to refine the spatial pose of the preprocessed video frame sequence data using a feature-matching-based cross-modal registration algorithm, resulting in refined video frame pose parameters. This can be achieved using a deep learning cross-modal feature embedding method, specifically including: extracting multi-scale 3D feature descriptors from the 3D geometric reference model, wherein the multi-scale 3D feature descriptors include 3D corner features, 3D edge features, and 3D patch features; and extracting multi-scale 2D feature descriptors from the preprocessed video frame sequence data, wherein the multi-scale 2D feature descriptors include 2D corner features, 3D edge features, and 3D patch features. Point features, two-dimensional edge features, and two-dimensional texture features are used. Based on the multi-scale three-dimensional feature descriptors and the multi-scale two-dimensional feature descriptors, a deep learning-based cross-modal feature embedding network is used to map the three-dimensional and two-dimensional features to a unified feature embedding space, resulting in three-dimensional embedded feature vectors and two-dimensional embedded feature vectors. Based on the three-dimensional and two-dimensional embedded feature vectors, a nearest neighbor matching-based cross-modal feature correspondence establishment method is used to determine the correspondence set between three-dimensional feature points and two-dimensional feature points. Based on the correspondence set and the initial spatial pose parameters, a pose estimation method based on random sampling consistency is used to perform outlier removal and pose refinement iterations to obtain refined video frame pose parameters.

[0141] It should be noted that multi-scale 3D feature descriptors are feature description vectors extracted from different scale levels of a 3D geometric model. 3D corner features refer to high-curvature points on the surface of a 3D model, such as eaves corners and ridge endpoints. 3D edge features refer to sharp edges on the surface of a 3D model, such as eaves outlines and door / window frames. 3D patch features refer to planar region features on the surface of a 3D model, such as wall planes and roof curves. Multi-scale 2D feature descriptors are feature description vectors extracted from different scale levels of video frame images. Cross-modal feature embedding networks are neural network architectures that map different modalities of data to a unified feature space through deep learning, making similar features from different modalities close in distance within the embedding space. Random sampling consistency is a robust estimation method that uses random sampling and model validation to eliminate outliers.

[0142] It is understandable that the features of 3D models and the features of video images belong to different representation spaces. Traditional matching methods based on handcrafted features, such as SIFT and ORB, experience a sharp decline in performance in cross-modal scenarios. This implementation uses a deep learning cross-modal feature embedding network to map 3D geometric features and 2D visual features to a unified embedding space, so that 3D roof corner and 2D roof corner images have similar vector representations in the embedding space, thereby achieving accurate cross-modal matching.

[0143] In actual implementation, the first step is to extract three-dimensional multi-scale features. On the three-dimensional geometric benchmark model, a Harris-3D-based corner detection algorithm is used to extract three-dimensional corner features, an edge detection and curvature analysis-based method is used to extract three-dimensional edge features, and a region segmentation and plane fitting-based method is used to extract three-dimensional patch features. For each three-dimensional feature point, its local shape descriptor, such as the SHOT descriptor or FPFH descriptor, is determined, resulting in a set of three-dimensional feature descriptors.

[0144] On the video frame image, a Harris-based corner detection algorithm is used to extract two-dimensional corner features, a Canny operator-based edge detection algorithm is used to extract two-dimensional edge features, and a Gabor filter-based texture analysis algorithm is used to extract two-dimensional texture features. For each two-dimensional feature point, its local appearance descriptor, such as the SIFT descriptor or ORB descriptor, is determined, resulting in a set of two-dimensional feature descriptors.

[0145] A cross-modal feature embedding network is constructed. This network contains two parallel encoder branches: a 3D feature encoder E3... D and two-dimensional feature encoder E 2D The two encoders employ a PointNet++ architecture to process unordered 3D feature point sets, while the 2D feature encoder uses a ResNet architecture to process image patches. The outputs of both encoders are mapped to a unified d-dimensional embedding space through a shared fully connected layer. During training, a contrastive loss function or a triplet loss function is used to ensure that matched 3D-2D feature pairs are close to each other in the embedding space, while mismatched feature pairs are far apart.

[0146] In the embedding space, for each 2D feature vector, the nearest neighbor 3D feature vector is found, establishing a set of correspondences between 3D and 2D feature points. Based on this set of correspondences and the initial spatial pose parameters, the RANSAC algorithm is used for pose estimation. A minimum sample set, such as 4 pairs of corresponding points, is randomly selected to determine the initial estimate of the camera pose. The reprojection error of all corresponding points is verified, and the pose estimate with the most inliers is retained. The pose parameters are refined through nonlinear optimization to obtain the refined video frame pose parameters.

[0147] In a preferred embodiment, to enrich the information dimensions of the registered monitoring data, the step of aligning the video frame sequence data with the digital twin baseline model at the pixel level according to the refined video frame pose parameters to obtain the registered monitoring data can be achieved using a three-channel cascade method. Specifically, this includes: determining the corresponding mapping relationship between video frame pixels and surface points of the three-dimensional model using a pixel coordinate mapping method based on projection transformation, based on the refined video frame pose parameters and the three-dimensional geometric reference model in the digital twin baseline model; projecting the multi-level semantic model of the building exterior in the digital twin baseline model onto the video frame image space to generate a semantic label image aligned with the video frame pixels; projecting the three-dimensional texture model data in the digital twin baseline model onto the video frame image space based on the corresponding mapping relationship to generate a reference texture image aligned with the video frame pixels; and cascading the video frame image, the semantic label image, and the reference texture image through channels to obtain the registered monitoring data, wherein the registered monitoring data includes video pixel information, semantic category information, and reference texture information.

[0148] It should be noted that projection transformation refers to the mathematical transformation that maps three-dimensional points to a two-dimensional image plane through a camera projection model. Semantic-labeled images are images where each pixel value represents the semantic category to which that pixel belongs, such as images of roofs, doors and windows, exterior walls, and decorative components. Baseline texture images are images where each pixel value represents the texture color of the corresponding location of that pixel in the digital twin baseline model. Channel concatenation refers to the operation of stitching multiple images together along the channel dimension to form a multi-channel tensor.

[0149] Understandably, traditional video analysis relies solely on raw pixel information for judgment, lacking an understanding of pixel semantic categories and a reference to baseline states. This application, through three-channel cascading, enables each video pixel to simultaneously carry three types of information: the current observation value, i.e., the video pixel; the semantic category, i.e., whether the pixel belongs to the roof / door / window / exterior wall / decorative component; and the baseline state, i.e., the normal texture of the pixel in the digital twin baseline, providing unprecedented information richness for subsequent anomaly detection.

[0150] In actual execution, based on the refined video frame pose parameters and the three-dimensional geometric reference model, for each video frame pixel, its corresponding three-dimensional ray is determined by inverse projection, and the intersection point of the ray with the surface of the three-dimensional model is calculated to obtain the corresponding three-dimensional surface point.

[0151] Based on the corresponding mapping relationship, query the multi-level semantic model of the building appearance, determine the semantic category c(u,v)∈{roof, doors and windows, exterior walls, decorative components} of the three-dimensional surface points, and generate semantic label images.

[0152] By querying 3D texture model data, the baseline texture color of 3D surface points is determined, and a baseline texture image is generated. The video frame image, semantic label image, and baseline texture image are concatenated along the channel dimension to obtain the registered monitoring data, providing rich information support for subsequent anomaly detection.

[0153] Step S60: Based on the registered monitoring data and the protected area vector data, conduct building anomaly detection and construction activity monitoring to obtain anomaly monitoring result data, and perform graded early warning and reliable evidence storage processing based on the anomaly monitoring result data to achieve the protection and monitoring of traditional villages.

[0154] It should be noted that building anomaly detection refers to the process of identifying areas where a building's appearance has changed or been damaged by comparing current video observations with a digital twin baseline. Construction activity monitoring refers to the process of detecting and identifying construction activities such as new construction, renovation, and demolition within a protected area. Anomaly monitoring results are fused data from building anomaly detection and construction activity monitoring. Tiered early warning refers to triggering different levels of early warning responses based on the severity and urgency of the anomaly. Trusted evidence storage refers to verifying the integrity of monitoring data associated with early warning events and distributing its storage to ensure the immutability of the evidence chain.

[0155] Understandably, traditional surveillance systems rely solely on changes in video pixels for anomaly detection, lacking semantic understanding and baseline references, leading to numerous false alarms. For example, pixel changes caused by lighting variations or swaying trees are misinterpreted as building anomalies. This implementation, through registered monitoring data, enables anomaly detection at the semantic level, including rooftops, doors and windows, exterior walls, and decorative components. It also allows for precise comparison using a digital twin baseline, significantly improving detection accuracy and reducing the false alarm rate.

[0156] This embodiment provides a method for the protection of traditional villages based on digital twin baselines. It acquires multi-source spatial data and performs preprocessing and coordinate registration to obtain spatial baseline framework data and registered multi-source data. Based on the spatial baseline framework data and the registered multi-source data, it extracts features of protected buildings and identifies value elements to obtain protected building object data. It then fuses the protected building object data, existing protected object data, and natural geographic element data to construct a spatial correlation map and performs protection priority learning and zoning boundary generation to obtain protected zoning vector data. Based on the protected building object data, it constructs a digital twin baseline model, which includes a multi-level semantic model of building appearance and a three-dimensional geometric baseline model. It acquires surveillance video stream data and performs cross-modal registration with the digital twin baseline model to obtain registered monitoring data. Based on the registered monitoring data and the protected zoning vector data, it performs building anomaly detection and construction activity monitoring to obtain anomaly monitoring result data. Finally, it performs tiered early warning and reliable evidence storage processing based on the anomaly monitoring result data to achieve the protection and monitoring of traditional villages. Through the above methods, by constructing a spatial correlation graph and using graph neural networks for protection priority learning, the objective and scientific generation of protection zoning boundaries has been achieved. Through cross-modal registration of digital twin baseline models and real-time monitoring videos, the accuracy of building hazard detection has been significantly improved and the false alarm rate has been reduced. Through the dynamic linkage between adaptive electronic fences and zoning results, data association and collaborative linkage between building identification, zoning generation and real-time monitoring and early warning in the protection of traditional villages have been realized. This has improved the efficiency of building identification, enabled the proactive detection of building hazards and illegal construction activities, enhanced the intelligence level and response time of traditional village protection, and prevented the destruction of the traditional village landscape.

[0157] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Based on this, step S60 includes steps S601 to S609: Step S601: Based on the registered monitoring data, extract the temporal optical flow field data between consecutive video frames.

[0158] It should be noted that temporal optical flow field data refers to a two-dimensional vector field describing the distribution of pixel motion vectors between consecutive video frames. Optical flow is the apparent motion of image brightness, reflecting the displacement changes of pixels in an image sequence over time. In architectural anomaly detection scenarios, optical flow fields can capture subtle dynamic changes in the building's appearance, including slow displacement caused by structural deformation, localized movement caused by crack propagation, and sudden displacement caused by component detachment. By analyzing the optical flow changes across multiple consecutive video frames, temporal optical flow fields can reveal the dynamic evolution of anomalous events, providing temporal evidence for subsequent anomaly identification.

[0159] Understandably, traditional village buildings are mostly timber-framed or brick-and-timber structures, which will produce slight structural responses under natural environmental factors such as temperature changes, humidity variations, and wind loads. Structural responses under normal conditions are regular and reversible, while structural responses under abnormal conditions, such as crack propagation and foundation settlement, are cumulative and irreversible. Through long-term monitoring of temporal optical flow fields, normal structural responses and abnormal structural changes can be distinguished, enabling early warning of building hazards.

[0160] In the specific implementation, a continuous video frame sequence is extracted from the registered monitoring data. ,in For the t-th frame image, it contains three channels: video pixel information, semantic category information, and baseline texture information. For each pair of adjacent frames ( , An optical flow calculation method based on the Lucas-Kanade algorithm is used to extract the optical flow field. The Lucas-Kanade algorithm is based on the constant brightness assumption and the spatial consistency assumption. The constant brightness assumption assumes that the brightness of the same pixel remains unchanged between adjacent frames, while the spatial consistency assumption assumes that adjacent pixels have similar motion. By applying smoothing constraints to the optical flow in the local neighborhood to solve for the optical flow vector, the dense optical flow field between each pair of adjacent frames is finally obtained, and the time-series optical flow field data covering the entire monitoring period is integrated.

[0161] It is worth noting that, to improve the accuracy and robustness of optical flow calculation, a pyramid-based hierarchical optical flow calculation method can also be adopted. First, a Gaussian pyramid is constructed for the image. At the top layer, i.e., the lowest resolution layer, coarse-grained optical flow estimation is performed. Then, the estimation results are passed to the next layer as initial values ​​for refinement, until the bottom layer, i.e., the original resolution layer. The pyramid hierarchical strategy can effectively handle large displacement motions while maintaining the calculation accuracy for fine-grained motions.

[0162] Let the number of pyramid layers be L, and the first... l Layer image is I (l) The resolution is 1 / 2 of the original image. l The optical flow is determined starting from the top layer L, and then propagated downwards layer by layer. The estimated optical flow results from the upper layers are used as the initial optical flow for the current layer. Iterative refinement is performed at the current layer resolution to obtain a more accurate optical flow estimate. Finally, a dense optical flow field with satisfactory accuracy is obtained at the original resolution layer. For construction activity scenarios involving significant large displacements, the pyramid-layered optical flow calculation method can significantly improve the accuracy of optical flow estimation and avoid computational failures caused by large displacements exceeding the local neighborhood search range.

[0163] To enhance robustness to changes in illumination, video frames are logarithmically transformed and normalized before optical flow calculation. Furthermore, to distinguish between building motion and external environment motion, optical flow calculation is performed only on pixels in the registered monitoring data that are semantically labeled as building areas, such as rooftops, doors, windows, exterior walls, and decorative components, filtering out motion interference from non-building areas such as the sky, ground, and vegetation.

[0164] Step S602: Based on the temporal optical flow field data, determine the structural similarity change value of each semantic region of the building exterior.

[0165] It should be noted that the structural similarity change value refers to a quantitative indicator that measures the change in the degree of structural similarity between the current observed state of each semantic region of the building's exterior and the digital twin baseline state. Structural similarity (SSIM) comprehensively considers the similarity across three dimensions: brightness, contrast, and structure. Compared to traditional pixel-level difference measures, such as mean square error (MSE), it better reflects the characteristics of human visual perception and has better robustness to changes in illumination and slight noise.

[0166] Understandably, abnormal changes in a building's appearance manifest not only as alterations in pixel values ​​but also as disruptions to local structural information. For example, cracks disrupt the continuity of a wall, and falling components alter the integrity of the outline. SSIM effectively captures this loss of structural information, rather than merely changes in color or brightness, by comparing the mean, standard deviation, and covariance of brightness within a local window.

[0167] In a preferred embodiment, to accurately detect abnormal changes in various semantic regions of the building exterior and filter out environmental interference, the determination of structural similarity change values ​​for each semantic region of the building exterior based on the temporal optical flow field data can employ a semantic-level motion separation detection method. Specifically, this includes: dividing the video frame image into a roof semantic region, a door and window semantic region, an exterior wall semantic region, and a decorative component semantic region based on the semantic tag images in the registered monitoring data; for each semantic region, calculating the optical flow vector amplitude and direction angle of each pixel within that semantic region based on the temporal optical flow field data; and performing motion consistency analysis on the pixels within that semantic region based on the optical flow vector amplitude and direction angle. Motion consistency clustering is used to distinguish between global motion caused by normal environmental disturbances and local motion caused by abnormalities in the building itself. Based on the motion consistency clustering results, global motion pixels are filtered out, and local motion pixels are retained to obtain a set of abnormal motion pixels. For each pixel in the set of abnormal motion pixels, the current pixel value in the current video frame and the reference pixel value in the reference texture image are extracted from the registered monitoring data. Based on the current pixel value and the reference pixel value, the local structural similarity value of the pixel is calculated. The local structural similarity values ​​of all abnormal motion pixels in the semantic region are statistically aggregated to obtain the structural similarity change value of the semantic region.

[0168] It should be noted that motion consistency clustering is a method of grouping pixels based on the similarity of their motion vectors; pixels with similar motion patterns are grouped into the same cluster. Global motion refers to the consistent motion of the entire image or a large area caused by environmental factors such as camera movement, changes in lighting, and weather changes. Local motion refers to abnormal motion in a local area caused by changes in the building itself, such as crack propagation or component detachment.

[0169] Understandably, environmental disturbances are extremely common in traditional village monitoring scenarios: wind blowing leaves, changes in lighting, and pedestrian movement can all cause significant pixel changes. Directly detecting all pixel changes would generate a large number of false alarms. This implementation uses motion consistency clustering to separate global environmental motion from local abnormal building motion, performing structural similarity analysis only on local motion areas. This effectively filters out environmental interference and significantly reduces the false alarm rate.

[0170] In practice, video frames are divided into four semantic regions based on semantically labeled images. For each semantic region, a temporal optical flow field data is extracted using an optical flow calculation method based on the Lucas-Kanade algorithm. Let the video frame at time t be... The video frame at time t+1 is Optical flow field To satisfy the constant brightness constraint, the optical flow vector of each pixel is solved by minimizing the sum of squared brightness residuals in the neighborhood. Calculate the optical flow vector magnitude for each pixel. and direction angle .

[0171] Motion-consistent clustering is performed. A density-based clustering algorithm using DBSCAN is employed, with optical flow amplitude and orientation angle as feature vectors. Define the neighborhood radius ϵ and the minimum number of samples MinPts. Pixels with reachable density are grouped into the same cluster. Global environment motion forms a large and dispersed cluster, while local building anomaly motion forms a small and dense cluster.

[0172] Global motion and local motion are distinguished by cluster size and density features. Let cluster C... k Contains N k Pixels, cluster center is Cluster density is ,in This represents the cluster coverage area. When N... k >N global and <ρ global If a pixel in the global motion cluster is selected, it is classified as a global motion cluster; otherwise, it is classified as a local motion cluster. All pixels in the global motion cluster are filtered out, and the pixels in the local motion cluster are retained to obtain the set of abnormal motion pixels. N global ρ is the preset threshold for the minimum number of pixels for global motion clustering. global The two parameters are preset global motion clustering maximum density thresholds and can be adjusted according to the actual monitoring range and image resolution of the monitoring scene.

[0173] For each pixel (x, y) in the set of abnormal motion pixels, extract the current pixel value and the reference pixel value from the registered monitoring data. Calculate the local structural similarity value:

[0174] in, and These are the mean values ​​of the current pixel value and the baseline pixel value within a local window centered at (x, y), respectively. and For the corresponding variance, For covariance, and is the stability constant.

[0175] The local structural similarity values ​​of all abnormally moving pixels within the semantic region are statistically aggregated to obtain the structural similarity change value within the semantic region. When the structural similarity change value is less than a preset threshold, such as 0.85, the semantic region is determined to be abnormal.

[0176] Step S603: Compare the structural similarity change value with a preset abnormal change threshold to obtain preliminary abnormal region identification data.

[0177] It should be noted that the abnormal change threshold is the cutoff value for determining whether a change in structural similarity constitutes an anomaly. The threshold setting directly affects the detection sensitivity and false alarm rate: a threshold that is too high leads to missed detections, while a threshold that is too low leads to false alarms. Methods for determining the threshold include statistical learning methods based on training data, methods based on expert experience, and adaptive threshold adjustment methods.

[0178] Understandably, different semantic regions have different structural characteristics and light sensitivity, making it difficult to adapt a uniform threshold to all regions. For example, the roof region typically has a higher structural similarity variation value due to the regular arrangement of tiles and strong texture repetition, so the threshold can be appropriately relaxed; while the decorative component region has a larger fluctuation in structural similarity variation value due to its complex details and sensitivity to local changes, so the threshold should be appropriately tightened.

[0179] Specifically, a threshold setting method based on semantic region type adaptation is adopted. An independent threshold is set for each semantic region type. The threshold setting is based on: the statistical distribution of historical structural similarity changes of the semantic region, the sensitivity of the region to structural changes, and the expert's tolerance for this type of anomaly.

[0180] The structural similarity change value of each semantic region is compared with the corresponding threshold to obtain a preliminary anomaly label. When the preliminary anomaly label is 1, the anomaly information of the semantic region is recorded, including: the location of the anomaly region, i.e., the range of pixel coordinates, the area of ​​the anomaly region, the structural similarity change value, the time of anomaly occurrence, etc., to obtain the preliminary anomaly region label data.

[0181] Step S604: Perform time-series consistency verification on the preliminary abnormal region identification data to obtain verified abnormal region data.

[0182] It should be noted that temporal consistency verification is a verification method that filters out false detections caused by transient interference by analyzing the consistency of anomaly detection results over time. Transient interference in surveillance videos includes: sudden changes in lighting, such as shadow changes caused by rapidly moving clouds, birds / insects flying past the camera, brief obstruction by leaves / flags, and vehicle headlights sweeping across the screen. The pixel changes caused by these interferences are transient and localized, unlike the continuous changes in architectural anomalies such as crack propagation and component detachment.

[0183] Understandably, transient interference is extremely common in traditional village monitoring environments. For example, a bird flying across the screen might cause significant pixel changes in a single frame, leading to a misjudgment as a building anomaly; however, if the area is tracked across multiple consecutive frames, the changes will quickly disappear, thus identifying it as a false detection. Temporal consistency verification effectively filters out transient interference by requiring the abnormal area to persist across multiple consecutive frames and exhibit a consistent trend of change.

[0184] In the specific implementation, timing consistency verification includes: establishing a timing anomaly tracking window. Let the current frame be t, and the window length be T. window A sliding time window is used to analyze the temporal behavior of the abnormal region within that window. The duration of the anomaly is determined; if the duration exceeds a threshold (e.g., 0.7), meaning the anomaly is detected in more than 70% of frames, it is considered to have temporal persistence. Furthermore, the consistency of the anomaly's change trend is determined. This involves performing a temporal analysis of the structural similarity changes within the anomaly region to determine the trend. When the anomaly is a genuine structural change, the trend should show a continuous decrease or a stable trend; when the anomaly is a transient disturbance, the trend should show a rapid recovery after sharp fluctuations. Based on these two dimensions—persistence and trend consistency—the initial anomalies are validated. False detections that are only transient or lack a persistent trend are eliminated, ultimately yielding validated stable anomaly region data, providing a reliable basis for subsequent protection decisions.

[0185] Step S605: Based on the verified abnormal area data, and combined with the multi-level semantic model of the building appearance, the abnormality type is determined to obtain the building abnormality detection result data.

[0186] It should be noted that anomaly type determination refers to identifying specific building anomaly types based on the semantic category, change characteristics, and spatial location of the anomaly area, such as exterior deformation, crack propagation, structural tilting, and component detachment. The multi-level semantic model of building exterior provides structured semantic information about anomaly areas, enabling anomaly type determination to transcend pixel-level analysis and perform refined diagnosis at the component level.

[0187] Understandably, anomaly types vary significantly across different semantic regions. For example, anomalies in the roof region might include tile detachment, ridge cracking, and drooping eaves; anomalies in the exterior wall region might include wall cracks, brick detachment, and peeling plaster; anomalies in the door and window region might include deformed door frames, broken window lattices, and broken glass; and anomalies in the decorative component region might include decayed brackets, broken corbels, and weathered plinths. Combining semantic models with anomaly type determination enables more precise anomaly localization and semantically descriptive anomaly descriptions.

[0188] In the specific implementation, based on the verified abnormal region data, the semantic level and specific component where the abnormality resides are determined. By querying the multi-level semantic model of the building exterior, the semantic tags and component information corresponding to the abnormal region are obtained. The morphological and textural features of the abnormal region are extracted. Morphological features include the outline shape, aspect ratio, principal axis direction, and boundary complexity of the abnormal region; textural features include the texture directionality, texture periodicity, and texture roughness within the abnormal region.

[0189] Morphological and textural features are input into a pre-trained anomaly type classifier. The anomaly type classifier employs a multi-classification model based on support vector machines or random forests, with training samples consisting of historically labeled anomaly cases. The output of the anomaly type classifier is an anomaly type probability distribution; the type with the highest probability is selected as the judgment result. Building anomaly detection results data include anomaly identifier, anomaly type, anomaly semantic location, anomaly pixel coordinates, actual anomaly area, anomaly severity level, anomaly occurrence time, and anomaly structural similarity change values.

[0190] Step S606: Construct an adaptive electronic fence in the registered monitoring data based on the protected area vector data.

[0191] It should be noted that an adaptive electronic fence is a virtual boundary area with an early warning level that is dynamically generated based on protected area vector data in the video image space. Unlike traditional electronic fences with fixed geometry, the boundary shape of an adaptive electronic fence precisely coincides with the actual protected area boundary and can adaptively update according to changes in camera pose.

[0192] Understandably, the management requirements of traditional village protection zones need to be reflected in the monitoring system in real time. Within the core protection area, any new construction, expansion, or alteration is generally prohibited; within the construction control zone, new construction and alteration that meet the style requirements are permitted, but must satisfy constraints such as height, form, and materials. Adaptive electronic fences translate the management requirements of protection zones into real-time spatial constraints in video surveillance. When a construction activity enters the corresponding area, the system automatically triggers the corresponding level of warning.

[0193] In a preferred embodiment, to achieve dynamic linkage between the protected area and the monitoring system, the construction of an adaptive electronic fence in the registered monitoring data based on the protected area vector data can employ a real-time geographic coordinate-image space conversion method. Specifically, this includes: acquiring the core protected area boundary line and the construction control zone boundary line from the protected area vector data; based on the spatial reference frame data, converting the core protected area boundary line and the construction control zone boundary line from geographic coordinate space to video frame image space to obtain the core protected area image boundary and the construction control zone image boundary; generating electronic fence areas with different warning levels in the video frame image based on the core protected area image boundary and the construction control zone image boundary, wherein the area inside the core protected area image boundary is a level one warning area, and the area between the core protected area image boundary and the construction control zone image boundary is a level two warning area; and monitoring the spatial pose changes of the camera in real time based on the refined video frame pose parameters, triggering a recalculation of the electronic fence area when the spatial pose change exceeds a preset threshold to obtain an adaptively updated electronic fence area.

[0194] It should be noted that geographic coordinate-image space transformation refers to the process of mapping the vector boundaries of a geographic coordinate system, such as WGS-84, to the pixel coordinate system of a video image through coordinate transformation and camera projection model. An electronic fence area refers to a virtual boundary area with a specific warning level defined in the video image space. Spatial pose change refers to changes in the position and attitude parameters of a camera in space, such as movement by external forces or swaying in the wind.

[0195] Understandably, traditional electronic fences are mostly fixed geometric areas, such as circles or rectangles, which are disconnected from the actual protected area boundaries. Furthermore, when the camera position changes, the electronic fence cannot adaptively adjust, leading to monitoring failure. This implementation projects the protected area vector data into the video image space in real time, ensuring the electronic fence precisely coincides with the planned boundary. Simultaneously, by monitoring changes in camera pose, it automatically triggers recalculation when the camera shifts, ensuring the electronic fence always maintains a spatial correspondence with the protected area.

[0196] In practice, the boundaries of the core protected area and the construction control zone in the protected area vector data are transformed from geographic coordinate space (longitude and latitude) to a unified coordinate system of the spatial reference framework data, such as UTM projection coordinates. Let the geographic coordinates be (λ, φ), which are then converted to planar coordinates (X, Y) using the Gauss-Kruger projection.

[0197] The planar coordinates are transformed into the video image space using a camera projection model. By traversing all vertices on the boundary lines, the core protection area boundary line and the construction control zone boundary line are projected into the video image space to obtain the corresponding image boundaries. In the image space, electronic fence areas are generated based on the projected boundary lines. The area inside the core protection area image boundary is marked as a Level 1 warning area, the area between the core protection area image boundary and the construction control zone image boundary is marked as a Level 2 warning area, and the area outside the construction control zone image boundary is marked as a Level 3 monitoring area.

[0198] Real-time monitoring of camera spatial pose changes. The amount of pose change is determined by comparing refined pose parameters across consecutive video frames. and .when >δ t such as 0.5 meters or >δ r For example, if the angle is 5°, it is determined that the camera pose has changed significantly, triggering the recalculation of the electronic fence area. The geographic coordinate-image space transformation process is re-executed to obtain the adaptively updated electronic fence area.

[0199] Step S607: Based on the adaptive electronic fence, perform construction activity target detection and time-series tracking on the registered monitoring data to obtain construction activity trajectory data.

[0200] It should be noted that construction activity target detection refers to the process of automatically identifying and locating construction activity-related targets within surveillance video frames. Construction activity-related targets include construction machinery, such as excavators, cranes, and concrete mixer trucks; building materials, such as sand and gravel piles, cement bags, and steel bar bundles; temporary facilities, such as scaffolding, fencing, and sheds; and construction workers. Temporal tracking refers to the continuous tracking of the same construction activity target within consecutive video frames, establishing the target's movement trajectory and behavioral sequence.

[0201] Understandably, illegal construction activities in traditional villages are often covert and sudden, such as secretly carrying out work at night or on holidays, or using small machinery for partial modifications. By automatically detecting and tracking the targets of construction activities over time, suspicious construction activities can be promptly identified and continuously monitored, providing a complete chain of evidence for subsequent determination of violations.

[0202] In the specific implementation, the target detection for the construction activity adopts a deep learning target detection algorithm based on YOLOv5. YOLOv5 is a single-stage target detection network with the characteristics of fast detection speed and high accuracy, which can meet the needs of real-time monitoring. The video pixel channels in the registered monitoring data are input into the pre-trained YOLOv5 detection network. The detection network outputs the bounding box coordinates (x, y, z) of the target. min ,ymin ,x max ,y max ), category label (class) and confidence score (confidence).

[0203] The categories of construction activity objectives include: construction equipment, such as excavators, cranes, concrete mixers, and transport vehicles; and building materials, such as sand and gravel stockpiles. pile Cement bags bag steel bars bar brick pile stack Temporary facilities, such as scaffolding, fences, and temporary sheds; construction workers, such as construction workers, can be identified by their safety helmets and reflective vests.

[0204] Temporal tracking employs a multi-target tracking algorithm based on DeepSORT. DeepSORT introduces depth appearance features into the SORT algorithm, achieving robust tracking of occluded and re-emerging targets through a combination of cascaded matching and IoU matching. The tracking process includes detection, prediction, matching, updating, and output. Detection involves detecting all construction activity targets in the current frame using YOLOv5. Prediction involves predicting the positions of existing tracked targets in the current frame using Kalman filtering. Matching involves associating the detection results with the predicted positions, including cascaded matching based on depth appearance features and IoU matching based on bounding box overlap. Updating involves updating the state of successfully matched tracked targets, such as position, velocity, and appearance features, initializing new unmatched targets, and deleting long-term unmatched targets. Output includes a unique ID, current position, motion trajectory, and historical behavior records for each tracked target.

[0205] The construction activity trajectory data includes: target ID, target category, target location sequence, target actual location sequence, target appearance time, target disappearance time, target movement speed, target activity area, etc.

[0206] Step S608: Based on the construction activity trajectory data, identify the violation behavior pattern to obtain construction activity monitoring result data.

[0207] It should be noted that violation behavior pattern recognition refers to the process of determining whether a construction activity violates the requirements of protected area zoning control based on changes in the geometric dimensions of the target, its temporal behavioral characteristics, and the semantic region category in which it is located. Violation behavior pattern recognition not only focuses on the existence of the target, but also on whether the target's behavioral patterns comply with regulations, achieving a leap from target detection to behavior understanding.

[0208] Understandably, single-dimensional features are insufficient to comprehensively characterize illegal construction activities. For example, detecting only height changes cannot distinguish between legal repairs and illegal additions; detecting only crowd gatherings cannot differentiate between normal residence and illegal construction. By integrating features from three dimensions—geometric dimensions, temporal behavior, and semantic regions—and matching them with a pre-defined rule base for determining illegal activities, accurate identification of illegal activities can be achieved.

[0209] In a preferred embodiment, to comprehensively identify violations during construction activities, the violation pattern recognition based on the construction activity trajectory data to obtain construction activity monitoring result data can employ a multi-dimensional feature fusion identification method. Specifically, this includes: extracting geometric dimension change features of the construction activity target based on the construction activity trajectory data, wherein the geometric dimension change features include changes in target height, target area, and target contour complexity; extracting temporal behavior features of the construction activity target based on the construction activity trajectory data, wherein the temporal behavior features include the frequency of construction machinery appearance, duration of building material stacking, and changes in personnel density; determining the semantic region category corresponding to the location of the construction activity target based on the semantic label image in the registered monitoring data; calling a preset violation judgment rule library according to the semantic region category, wherein the violation judgment rule library includes construction behavior prohibition rules within the core protection area, construction height restriction rules within the construction control zone, and construction form constraint rules; matching the geometric dimension change features and the temporal behavior features with the violation judgment rule library, and when the matching degree exceeds a preset threshold, determining the violation type to obtain the construction activity monitoring result data.

[0210] It should be noted that geometric dimension change characteristics refer to quantitative indicators describing the spatial scale changes of construction activity targets. Target height change refers to the vertical dimensional change of the construction activity target, such as the increase in the height of a new building. Target area change refers to the change in the projected area of ​​the construction activity target on the horizontal plane. Target contour complexity change refers to the change in the complexity of the construction activity target's contour boundary, such as changing from a simple rectangle to a complex polygon. Temporal behavior characteristics describe the behavioral patterns of construction activities evolving over time. Construction equipment frequency refers to the number of times construction equipment such as excavators and cranes are detected within a specific time period. Building material stockpiling duration refers to the continuous duration for building materials such as sand, gravel, and cement to be stockpiled on site. Personnel density change refers to the temporal trend of the number of personnel in the construction area.

[0211] Understandably, single-dimensional features are insufficient to comprehensively characterize illegal construction activities. For example, detecting only height changes cannot distinguish between legal repairs and illegal additions; detecting only gatherings of people cannot distinguish between normal residence and illegal construction. This implementation method achieves accurate identification of illegal activities by integrating features from three dimensions: geometric dimensions, temporal behavior, and semantic regions, and matching them with a pre-defined rule base for determining illegal activities.

[0212] In actual implementation, based on construction activity trajectory data, a monocular vision-based height measurement method using depth estimation is employed to determine the change in target height. A trained monocular depth estimation network estimates the depth value of each pixel from video frames. Combining this with the pixel height and depth values ​​of the target detection box, the actual height of the target is determined. The change in the target's footprint is determined by the pixel count and depth value of the target segmentation mask. Finally, the change in target contour complexity is determined by the ratio of contour perimeter to area.

[0213] Based on construction activity trajectory data, a pre-trained equipment detection model is used to identify excavators, cranes, mixers, etc., and the frequency of construction equipment occurrence is counted. The duration of building material stacking is determined through material area segmentation and temporal correlation analysis, and changes in personnel density are determined through a crowd density estimation network.

[0214] The semantically labeled image is queried to determine the semantic region category of the location of the construction activity target. If the target is located in a Level 1 warning zone, the construction prohibition rules within the core protection area are applied, meaning any new construction, expansion, or alteration is illegal. If the target is located in a Level 2 warning zone, the building height restriction rules and building form constraint rules are applied, meaning the building height must not exceed the height limit, the roof form must maintain a traditional style, and the facade materials must meet traditional material requirements.

[0215] The geometric dimension change features and temporal behavior features are matched with the violation judgment rule base. A violation matching degree function is defined as follows:

[0216] in, This refers to the deviation of geometric features from the rule threshold, such as the proportion of height exceeding the limit. The similarity between temporal behavior and violation patterns is used, such as scoring based on the high frequency of device occurrence. The score is the degree of matching between the semantic region and the applicability of the rule. For example, construction within the core protection area scores 1. This refers to the rule trigger strength, i.e., the number of rules that are triggered. , , These are the weighting coefficients.

[0217] when >τ violationWhen violations are identified, the type of violation is determined, such as violations related to excessive height, formal violations, or material violations, and the monitoring results data of the construction activities are obtained.

[0218] Step S609: Integrate the building anomaly detection result data and the construction activity monitoring result data to obtain anomaly monitoring result data, and perform graded early warning and reliable evidence storage processing based on the anomaly monitoring result data to achieve the protection and monitoring of traditional villages.

[0219] It should be noted that the anomaly monitoring results data is a fusion of building anomaly detection results and construction activity monitoring results, containing comprehensive information on both types of events. Tiered early warning refers to triggering different levels of early warning responses based on the severity and urgency of the anomaly. Trusted evidence storage refers to performing integrity verification and distributed storage on the monitoring data associated with the early warning event to ensure the integrity, immutability, and traceability of the evidence chain.

[0220] It is understandable that there may be a causal relationship between building anomalies and construction violations. For example, illegal construction activities may cause vibration damage or foundation settlement of adjacent protected buildings; structural anomalies may prompt owners to carry out unauthorized repairs. By integrating the monitoring results of these two types of events, potential causal relationships can be discovered, enabling more comprehensive risk assessment and more accurate early warning and response.

[0221] In a preferred embodiment, to ensure the reliability of the early warning information and the integrity of the evidence chain, the step of performing graded early warning and credible evidence storage based on the abnormal monitoring results data to achieve the protection and monitoring of traditional villages can adopt a comprehensive processing method based on decision matrix and hash chain. Specifically, this includes: determining the building anomaly early warning level based on the anomaly type and severity in the building anomaly detection results data; determining the construction activity early warning level based on the type of violation and the degree of violation risk in the construction activity monitoring results data; and determining the comprehensive early warning level using a comprehensive early warning level determination method based on decision matrix, based on the building anomaly early warning level and the construction activity early warning level. Based on the comprehensive early warning level, a corresponding early warning response strategy is matched, including an information push strategy, an on-site verification strategy, and an emergency response strategy. The associated video clip data and associated spatial coordinate data that triggered the early warning are extracted, and a data integrity verification code is generated using a hash chain-based time-series data integrity verification method. The associated video clip data, the associated spatial coordinate data, the data integrity verification code, and the early warning timestamp are bound and encapsulated to obtain a trusted evidence storage data packet. The trusted evidence storage data packet is uploaded to a distributed evidence storage node network to obtain an evidence storage hash value, and the evidence storage hash value is associated with the comprehensive early warning level for storage, thereby achieving the protection and monitoring of traditional villages.

[0222] It should be noted that a decision matrix is ​​a structured method for decision-making through multi-factor weighted scoring. A comprehensive early warning level is a unified early warning level determined by comprehensively considering the severity of two types of events: building anomalies and construction violations. An early warning response strategy refers to the set of specific handling measures determined based on the early warning level. A hash chain is a chain-like data structure formed by concatenating the hash values ​​of adjacent data blocks to create an immutable chain. A data integrity check code is a fixed-length digest value calculated using a hash algorithm to verify data integrity. A distributed evidence storage node network refers to a distributed storage network composed of multiple independent nodes that ensures data consistency through a consensus mechanism.

[0223] Understandably, traditional monitoring systems often lack prioritization in their early warning information, resulting in a large number of low-level warnings overwhelming important emergencies and leading to inefficient management responses. Furthermore, evidence data related to early warning events is easily tampered with or lost. This application uses a decision matrix to comprehensively determine the early warning level, ensuring priority response to emergencies; and employs hash chains and distributed evidence storage to ensure the integrity and immutability of the evidence data.

[0224] In practice, the building anomaly warning level is determined based on the anomaly type and severity. Let the weight of the anomaly type be w. type The structural tilt > crack propagation > external deformation, with an anomaly severity level of s. severity The building anomaly warning levels are determined by the proportion of abnormal area and the deviation of SSIM, ranging from 1 to 5, with level 1 being the lowest and level 5 being the highest.

[0225] The early warning level for construction activities is determined based on the type of violation and the degree of risk associated with it. Let w be the weight for each violation type. violation Within the core protection area, the violation is ranked as follows: > excessive height violation > formal violation > material violation, with a violation risk level of r. risk Based on the estimation of construction progress and scale, the early warning level for construction activities is also divided into 1-5 levels, with level 1 being the lowest and level 5 being the highest. A comprehensive early warning decision matrix with two rows and five columns is constructed. The rows correspond to the early warning level for building anomalies and the early warning level for construction activities, respectively, and the columns correspond to the weighted scores of different levels under the corresponding dimension. The final comprehensive early warning level is obtained by rounding up the weighted average of the scores of the two dimensions.

[0226] The corresponding response strategy is matched according to the comprehensive early warning level: Level 1-2 triggers the information push strategy; Level 3 triggers the on-site verification strategy; Level 4-5 triggers the emergency response strategy.

[0227] Extract the associated video clips that triggered the early warning (30 seconds before and after the warning) and the associated spatial coordinates (3D coordinates of the abnormal or illegally constructed area). A hash chain based on SHA-256 is used for temporal data integrity verification. The associated video clip data, associated spatial coordinates, data integrity check code, and warning timestamp are bound and encapsulated. The trusted evidence data packet is uploaded to a distributed evidence storage node network, such as a blockchain-based distributed storage system. Each evidence storage node independently calculates the hash value of the data packet, and after confirmation through a consensus mechanism, the evidence hash value is obtained. The evidence hash value is associated with the comprehensive early warning level and stored to form a complete early warning-evidence record, achieving closed-loop management for the protection of traditional villages.

[0228] In this embodiment, refined identification of building appearance anomalies is achieved through temporal optical flow field analysis and structural similarity change detection; false detections caused by environmental interference are effectively filtered out through motion consistency clustering and temporal consistency verification; real-time monitoring of construction activities and accurate identification of violations are achieved through the dynamic linkage between adaptive electronic fences and protected zoning; and closed-loop management of the entire chain from anomaly detection to early warning response and evidence preservation is achieved through hierarchical early warning and reliable evidence storage, significantly improving the intelligence level and response timeliness of traditional village protection.

[0229] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the traditional village protection method based on digital twin baselines in this application. Any simple modifications based on this technical concept are within the scope of protection of this application.

[0230] This application also provides a traditional village protection device based on a digital twin baseline; please refer to... Figure 2 Traditional village protection devices based on digital twin baselines include: The registration module 10 is used to acquire multi-source spatial data and perform preprocessing and coordinate registration to obtain spatial reference frame data and registered multi-source data.

[0231] The identification module 20 is used to extract the features of protected buildings and identify their value elements based on the spatial reference frame data and the registered multi-source data, so as to obtain protected building object data.

[0232] The construction module 30 is used to integrate the protected building object data, existing protected object data and natural geographic element data, construct a spatial association map and perform protection priority learning and zoning boundary generation to obtain protected zoning vector data.

[0233] The construction module 30 is also used to construct a digital twin baseline model based on the protected building object data, wherein the digital twin baseline model includes a multi-level semantic model of the building appearance and a three-dimensional geometric benchmark model.

[0234] The registration module 10 is also used to acquire monitoring video stream data, and to perform cross-modal registration of the monitoring video stream data with the digital twin baseline model to obtain registered monitoring data.

[0235] The monitoring module 40 is used to detect building anomalies and monitor construction activities based on the registered monitoring data and the protected area vector data, obtain anomaly monitoring result data, and perform graded early warning and reliable evidence storage processing based on the anomaly monitoring result data, so as to realize the protection and monitoring of traditional villages.

[0236] The traditional village protection device based on digital twin baselines provided in this application, employing the traditional village protection method based on digital twin baselines in the above embodiments, can solve the technical problems of low efficiency in identifying protected buildings through manual surveys in existing traditional village protection, and the passive and delayed monitoring and early warning systems that fail to proactively detect building hazards and illegal construction activities, leading to damage to the traditional village landscape. Compared with the prior art, the beneficial effects of the traditional village protection device based on digital twin baselines provided in this application are the same as those of the traditional village protection method based on digital twin baselines provided in the above embodiments, and other technical features in the traditional village protection device based on digital twin baselines are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0237] This application also proposes a traditional village protection device based on a digital twin baseline, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the traditional village protection method based on a digital twin baseline as described above.

[0238] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the traditional village protection method based on digital twin baselines as described above.

[0239] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for the preservation of traditional villages based on digital twin baselines, characterized in that, The method includes: Acquire multi-source spatial data and perform preprocessing and coordinate registration to obtain spatial reference frame data and registered multi-source data; Based on the spatial reference framework data and the registered multi-source data, the protected building features are extracted and the value elements are identified to obtain protected building object data. By integrating the data of the protected building objects, existing protected object data, and natural geographic element data, a spatial relationship map is constructed, and protection priority learning and zoning boundary generation are performed to obtain protected zoning vector data; A digital twin baseline model is constructed based on the data of the protected building object, wherein the digital twin baseline model includes a multi-level semantic model of the building appearance and a three-dimensional geometric benchmark model. Acquire surveillance video stream data, and perform cross-modal registration of the surveillance video stream data with the digital twin baseline model to obtain registered surveillance data; Based on the registered monitoring data and the protected area vector data, building anomaly detection and construction activity monitoring are carried out to obtain anomaly monitoring result data. Based on the anomaly monitoring result data, graded early warning and reliable evidence storage are carried out to achieve the protection and monitoring of traditional villages.

2. The method as described in claim 1, characterized in that, The process of acquiring multi-source spatial data, performing preprocessing and coordinate registration to obtain spatial reference frame data and registered multi-source data includes: Acquire multi-source spatial data, wherein the multi-source spatial data includes at least remote sensing image data, ground image data, and basic geographic information data; The remote sensing image data is subjected to radiometric and geometric correction processing, the ground image data is subjected to distortion correction and illumination enhancement processing, and the basic geographic information data is subjected to coordinate system 1 and format standardization processing to obtain preprocessed multi-source data. A unified spatial coordinate reference system is established based on the preprocessed multi-source data to obtain spatial reference frame data. Based on the spatial reference framework data, spatial coordinate registration is performed on the preprocessed multi-source data to obtain the registered multi-source data.

3. The method as described in claim 1, characterized in that, The process of extracting features and identifying value elements of protected buildings based on the spatial reference framework data and the registered multi-source data to obtain protected building object data includes: Based on the registered multi-source data, the morphological and color feature parameters of the building roof are extracted. Based on the preset threshold values ​​for protected building characteristics, the morphological feature parameters and the color feature parameters are jointly screened to obtain a preliminary set of building data. For each building in the initial screening of the building set data, extract local image data of the building from the registered multi-source data; Based on the aforementioned partial building image data, a fine-grained visual recognition model is used to detect value elements, and value element detection result data is obtained. Value is determined based on the value element detection results data, buildings with value elements are identified as protected buildings, and the spatial location information of each protected building is determined based on the spatial reference frame data to obtain protected building object data.

4. The method as described in claim 1, characterized in that, The process involves integrating the data of the protected building objects, existing protected object data, and natural geographic element data to construct a spatial relationship map, and then performing protection priority learning and zoning boundary generation to obtain protected zoning vector data, including: Acquire existing protected object data and natural geographic element data, wherein the existing protected object data includes at least immovable cultural relic spatial data and historical environmental element spatial data, and the natural geographic element data includes at least road data, water system data and topographic data; By integrating the data of the protected building objects, the data of existing protected objects, and the data of natural geographic elements, a multi-source spatial dataset is constructed. Based on the multi-source spatial dataset, the spatial adjacency and topological connection relationships of each element node are extracted to construct a spatial association graph; Based on the spatial association graph, a graph neural network is used to learn the node protection priority embedding to obtain the protection priority vector of each node. Based on the protection priority vector of each node, the spatial distribution density of protection elements is calculated to obtain the protection element density field data; Based on the density field data of the protected elements and the data of the natural geographic elements, the zoning boundary is generated to obtain the protected zoning vector data.

5. The method as described in claim 1, characterized in that, The construction of a digital twin baseline model based on the protected building object data includes: Extract architectural attribute information and spatial location information of each protected building from the protected building object data; Based on the building attribute information and the spatial location information, a three-dimensional geometric reconstruction is performed to obtain three-dimensional geometric model data; The three-dimensional geometric model data is subjected to appearance texture mapping to obtain three-dimensional texture model data; Based on the three-dimensional texture model data, a multi-level semantic model of building appearance is constructed, wherein the multi-level semantic model of building appearance includes at least a roof semantic layer, a door and window semantic layer, an exterior wall semantic layer, and a decorative component semantic layer. The three-dimensional texture model data is associated and bound with the multi-level semantic model of the building appearance to obtain a digital twin baseline model.

6. The method as described in claim 1, characterized in that, The process of acquiring surveillance video stream data and performing cross-modal registration between the surveillance video stream data and the digital twin baseline model to obtain registered monitoring data includes: Acquire surveillance video stream data, wherein the surveillance video stream data includes timestamp information and spatial location information of the acquisition device; The monitoring video stream data is subjected to keyframe extraction and quality enhancement processing to obtain preprocessed video frame sequence data; Based on the spatial location information of the acquisition device and the spatial reference frame data, the initial spatial attitude parameters of each video frame are determined. Based on the three-dimensional geometric reference model in the digital twin baseline model, a cross-modal registration algorithm based on feature matching is used to refine the spatial pose of the preprocessed video frame sequence data, thereby obtaining the refined video frame pose parameters. Based on the refined video frame pose parameters, the video frame sequence data is spatially aligned with the digital twin baseline model at the pixel level to obtain the registered monitoring data.

7. The method as described in claim 1, characterized in that, The process of detecting building anomalies and monitoring construction activities based on the registered monitoring data and the protected area vector data yields anomaly monitoring results data. Furthermore, tiered early warning and reliable evidence storage are performed based on these anomaly monitoring results data to achieve the protection and monitoring of traditional villages. This includes: Based on the registered monitoring data, temporal optical flow field data between consecutive video frames are extracted; Based on the temporal optical flow field data, the structural similarity change values ​​of each semantic region of the building exterior are determined; The structural similarity change value is compared with a preset abnormal change threshold to obtain preliminary abnormal region identification data; The temporal consistency of the preliminary abnormal region identification data is verified to obtain the verified abnormal region data. Based on the verified abnormal area data, the abnormality type is determined by combining the multi-level semantic model of the building appearance, and the building abnormality detection result data is obtained. Based on the protected area vector data, an adaptive electronic fence is constructed in the registered monitoring data; Based on the adaptive electronic fence, the registered monitoring data is used to detect construction activity targets and track time series to obtain construction activity trajectory data. Based on the construction activity trajectory data, violation behavior patterns are identified to obtain construction activity monitoring result data; By integrating the building anomaly detection results data and the construction activity monitoring results data, anomaly monitoring results data is obtained. Based on the anomaly monitoring results data, graded early warning and reliable evidence storage are carried out to achieve the protection and monitoring of traditional villages.

8. A traditional village protection device based on a digital twin baseline, characterized in that, The device includes: The registration module is used to acquire multi-source spatial data and perform preprocessing and coordinate registration to obtain spatial reference frame data and registered multi-source data. The identification module is used to extract the features of protected buildings and identify their value elements based on the spatial reference frame data and the registered multi-source data, so as to obtain protected building object data. The construction module is used to integrate the data of the protected building objects, existing protected object data and natural geographic element data, construct a spatial relationship map, and perform protection priority learning and zoning boundary generation to obtain protected zoning vector data; The construction module is also used to construct a digital twin baseline model based on the protected building object data, wherein the digital twin baseline model includes a multi-level semantic model of the building appearance and a three-dimensional geometric benchmark model. The registration module is also used to acquire monitoring video stream data, and to perform cross-modal registration of the monitoring video stream data with the digital twin baseline model to obtain the registered monitoring data. The monitoring module is used to detect building anomalies and monitor construction activities based on the registered monitoring data and the protected area vector data, obtain anomaly monitoring result data, and perform graded early warning and reliable evidence storage processing based on the anomaly monitoring result data, so as to realize the protection and monitoring of traditional villages.

9. A traditional village protection device based on a digital twin baseline, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the traditional village protection method based on a digital twin baseline as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the traditional village protection method based on digital twin baselines as described in any one of claims 1 to 7.

Images