A Narrative Method for Historical Maps Based on Augmented Reality

Abstract

The present invention discloses a historical map narrative method based on augmented reality, belonging to the fields of cartography and augmented reality technology. The method of the present invention comprises S1: acquiring raw data; S2: performing HGIS digitization processing on the acquired raw data; S3: saving the data processed in S2; and S4: constructing a historical narrative platform based on augmented reality technology to realize map narrative. The present invention embeds multimedia data and digital maps into an AR environment in an interactive form for historical narrative, which makes up for the shortcomings of traditional maps that are mainly used for geographic data visualization and spatial analysis. The present invention provides a portable technical platform and map narrative template for historical map narrative, lowering the technical threshold for AR map compilation and historical memory and dissemination.

Description

Technical Field

[0001] The present invention belongs to the technical fields of cartography, narratology, historical GIS, and augmented reality, and specifically relates to a historical map narrative method based on augmented reality. Background Art

[0002] Maps, as a spatial visualization language, are core tools for presenting geographic information and disseminating spatial knowledge. They play a vital role in reconstructing historical space and disseminating historical memory. Historical geographic information systems (HGIS) can migrate traditional maps (old maps) and historical geographic data onto a geospatial information infrastructure (GII) with unified temporal and spatial coordinates. This allows for the reproduction of lost historical scenes in the form of digital maps, serving as a digital platform for recreating historical time and space and preserving historical memory. Despite this, current HGIS map reproduction is primarily focused on the spatialization and analysis of historical information, rather than on universal map production and cultural heritage dissemination. Current map presentation technologies, such as narrative mapping, pan-mapping, and scenographics, do not specifically consider the narratives inherent in historical maps, lacking effective methods for narratively expressing history through historical maps.

[0003] Augmented Reality (AR) is a digital technology that blends the real and the virtual, overlaying digital content onto physical objects (such as paper maps) or locations in the real world. AR technology combines the real and virtual, integrates online and offline resources, enables real-time interaction, and registers objects in 3D. Key technologies include image recognition, plane recognition, and SLAM. Creating AR content on maps based on image recognition can effectively enhance the map's expressive dimensions and expand its scope and application.

[0004] AR map background technology currently involves the following aspects. First, during the map construction and data processing process, map data acquisition and processing requires the integration of computer vision and image processing technologies to perform data preprocessing, feature extraction, and matching operations to provide basic data support for AR map construction. However, it is necessary to solve the problem of fusing geospatial data from different sources and formats to fully and accurately construct an AR map database.

[0005] Secondly, the most important technology involved in geographic information enhancement and visualization is augmented reality overlay technology. Based on position and orientation sensor data, virtual geographic information elements (such as annotations, icons, and three-dimensional models) are precisely overlaid on corresponding locations in the real world, thereby enhancing and enriching the real scene. AR provides a new method for visualizing geographic spatial data, namely, a way to integrate virtual information into the real scene. Examples include the AR expression framework for locatable visual objects, projected augmented map systems, and the research framework for AR visualization systems. However, the core issues of augmented reality visualization are the fusion effect of the visualized objects with the real world and the rationality of the abstract information. Current research still pursues realistic effects and diverse expressions from the perspective of visualization, and relatively little research has been conducted on the rationality of abstract information. As a result, AR visualization lacks purpose and specificity.

[0006] Current augmented reality (AR) map narratives combine virtual information and real-world scenes, but their technical implementation and practical application still face numerous challenges. For example, user experience can suffer from information overload (excessive virtual tags obstructing vision), unnatural interactions (gesture / voice recognition delays), and nausea caused by prolonged use. Furthermore, users vary widely in their acceptance and understanding of the complexity of AR interfaces. Historical map narratives based on augmented reality (AR) represent a further development of visualization tools that combine spatial data with storytelling. Their core goal is to guide users through an immersive narrative experience using maps. However, these efforts face challenges in naturally integrating linear or nonlinear storylines with geospatial information, balancing information density with narrative pacing, avoiding resource loading delays or format conflicts when integrating multimedia content, and ensuring universal applicability across multiple cultural contexts. Furthermore, AR maps are currently not widely available, and their functionality is limited to route navigation. Few AR map products specifically designed for historical narratives exist, and a universal technical approach for AR-based historical map narratives is lacking. Summary of the Invention

[0007] In response to the problems mentioned in the background technology, the present invention proposes a historical map narrative method based on augmented reality (AR), applies HGIS technology to universal map production, and incorporates AR technology. Through this method, a universal framework is provided for the production of AR maps, which lowers the production threshold of historical map narratives and breaks the communication threshold between cartographers and map users.

[0008] Technical solution: In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0009] A historical map narrative method based on augmented reality includes the following steps:

[0010] S1: Get raw data;

[0011] S2: Perform HGIS digital processing on the acquired raw data;

[0012] S3: Save the data processed by S2;

[0013] S4: Build a historical narrative platform based on augmented reality technology to realize map narrative, specifically:

[0014] S41: Reproduce planning scenarios based on geographic databases and geographical environments;

[0015] S42: Present content in planned scenarios based on thematic geographic databases, multimedia libraries, and image recognition;

[0016] S43: Interactive settings and experience based on gesture recognition and content presentation;

[0017] S44: Manage and update content based on changed assets, interactive settings, and experiences to create map narratives.

[0018] Preferably, in S1, multi-source data collection is performed, including historical maps, historical documents and multimedia data.

[0019] Preferably, in S2, the specific process of performing HGIS digital processing on the acquired raw data is as follows:

[0020] S21: Perform HGIS digitization of historical maps, including:

[0021] Map preprocessing, map registration, ancient and modern verification, map vectorization and attribute assignment;

[0022] S22: Perform HGIS digitization of historical documents, including:

[0023] Text digitization, keyword extraction, geographic information spatialization and attribute association;

[0024] S23: Perform HGIS digital processing on multimedia data, including:

[0025] Data collection, format conversion and data compression.

[0026] As a preference, in S3, data storage: the specific contents of saving the data processed by S2 are:

[0027] The historical maps digitized by HGIS are input into the basic geographic database; the historical document data digitized by HGIS are input into the thematic geographic database; and the multimedia data digitized by HGIS are input into the multimedia material library.

[0028] Preferably, in S42, image recognition technology is used to load the target image and feature template, OpenCV's ORB algorithm is used for feature extraction, brute force matching is performed on the features, the 3D model is loaded and positioned, and the center of the model and the image are bound.

[0029] Preferably, in S42, the ORB algorithm performs feature extraction including:

[0030] In terms of spatial position, the FAST algorithm is used to detect corner points in the image;

[0031] In terms of direction attributes, the intensity centroid method is used to calculate the main direction of the key point, and the centroid of the feature point neighborhood is calculated using the geometric moment formula. The feature point and the centroid are connected to form the main direction. The geometric moment formula is calculated as follows:

[0032] ,

[0033] Among them, I(x,y) represents the pixel gray value of the image at point (x,y), and the centroid coordinates are , represents geometric moments; 、 Represents the power operation of pixel coordinates x and y, where p and q refer to the orders of the geometric moments; Indicates the azimuth of the centroid relative to the feature point; represents the component of the first-order moment in the x direction, represents the component of the first-order moment in the y direction; represents the zero-order moment, represents the four-quadrant inverse tangent function;

[0034] In terms of scale information, multi-scale detection is achieved through image pyramid.

[0035] Preferably, in terms of spatial position, the specific content of detecting corner points in an image by using the FAST algorithm is as follows:

[0036] Select candidate pixels: traverse each pixel in the image to check whether it is a focus candidate; take the pixel as the center and select 16 equally spaced circular pixels with a radius of 3 pixels to define its neighborhood range;

[0037] Quick pre-screening: Set a brightness difference threshold and check the four points numbered 1, 5, 9, and 13 on the circumference. If the absolute value of the brightness difference of three or more points is greater than the threshold, the pixel may be a corner point and proceed to the next step. Otherwise, skip directly.

[0038] Continuous pixel detection: Find a continuous series of N points among the 16 points on the circumference that meet one of the conditions of being brighter or darker than the center, and determine that the pixel is a corner point;

[0039] Non-maximum suppression: eliminates repeated corner points in dense areas and retains the most significant corner points.

[0040] As a preferred embodiment, in S42, the brute force matching rule based on the ORB feature is used, and its core matching rule is the full comparison principle. The specific matching process is as follows:

[0041] Distance metric: Improved Hamming distance is used to calculate the number of bits of difference between binary descriptors through XOR operation;

[0042] Nearest neighbor criterion: For each descriptor in the query set, traverse all descriptors in the training set and select the matching pair with the smallest distance as the candidate;

[0043] Threshold filtering: Eliminate fuzzy matches by setting a maximum distance threshold or using the nearest neighbor ratio method;

[0044] Cross-validation: Keep the matching pairs that are each other's nearest neighbors.

[0045] Preferably, in S42, the camera pose is solved according to the correspondence between the model coordinate system and the image coordinate system;

[0046] By binding the model to the scene center, the vertex coordinates of the 3D model are scaled to match the scale of the scene, ensuring that the model is aligned with the image after projection. Subsequently, the center point of the model is bound to the center of the image, and the model position is adjusted through translation transformation. The specific calculation formula is:

[0047] ,

[0048] Among them, K represents the camera internal parameter, Indicates the coordinates of the center of the image; Represents the adjusted translation vector; t represents the translation vector; project() represents the projection function; R represents the rotation matrix, Indicates the center point of the projected model.

[0049] Preferably, in S43, the gesture data is first preliminarily cleaned and converted, and the gesture recognition technology is used to obtain the screen gesture contact point to determine whether it intersects with the interactive object. If so, the corresponding event or action is triggered according to the predetermined logic to implement the response processing, and the user is provided with feedback on the successful operation through visual effects and sounds. The key nodes are:

[0050] Intersection detection loop: implement traversal detection of all contacts and UI elements through nested loops;

[0051] Conflict handling logic: When multiple touchpoints interact with an element, the priority strategy is used to filter the only valid interaction;

[0052] Gesture type matching: Mapping to predefined gesture types based on direction, speed, and trajectory characteristics;

[0053] Feedback mechanism: Visual highlights and sound prompts are triggered synchronously to enhance the user experience.

[0054] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0055] (1) For the first time, multimedia data (video, audio, web pages, text, etc.) and digital maps are embedded in an interactive form into the AR environment for historical narrative, making up for the shortcomings of traditional maps that are mainly used for geographic data visualization and spatial analysis.

[0056] (2) In the present invention, AR technology is combined to explore a new method of historical map narrative, and the environment setting, image recognition, and gesture recognition logic of the AR platform are encapsulated and adapted. Compared with the current two-dimensional solution based on the website, the technical solution of the present invention has stronger narrative and dissemination properties.

[0057] (3) The present invention provides a portable technical platform and map narrative template for historical map narratives, while supporting real-time interaction between cartographers and users, lowering the technical threshold for AR map compilation and historical memory dissemination, and increasing the speed of product updates and iterations.

[0058] (4) The feature extraction and matching implemented by the present invention based on the ORB algorithm can achieve more than 80% accurate matching and recognition in actual product scenarios (scores under the Vuforia platform), and achieves precise model positioning under user movement and different gestures, and is smooth and unobstructed in actual applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flow chart of the historical map narrative method based on augmented reality of the present invention;

[0060] Figure 2 It is the pseudo code logic diagram of the gesture recognition of the present invention;

[0061] Figure 3 Schematic diagram of the area that can be scanned in the practice of the present invention, wherein (a) is a front map and (b) is a back map;

[0062] Figure 4 It is a schematic diagram of organizing the geographical map narrative of the present invention;

[0063] Figure 5 This is a schematic diagram of the model matching and recognition accuracy obtained by optimizing the ORB algorithm in the present invention (the verification platform is Vuforia). DETAILED DESCRIPTION

[0064] The present invention will be further illustrated below with reference to specific examples. The examples are implemented based on the technical solutions of the present invention. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0065] The historical map narrative method based on augmented reality provided in this embodiment mainly includes the following steps:

[0066] S1: Get raw data;

[0067] Collect historical maps, historical documents and multimedia data.

[0068] The data sources of historical maps include 1:10,000 aerial topographic maps of a certain area in a certain historical period, "Architectural Maps of a Certain City in a Certain Historical Period", and Bing Maps, covering relevant content such as the places where historical events occurred, former residences of celebrities, and contemporary architecture.

[0069] The data sources of historical documents include "Diary of Someone", which covers specific content such as the time when historical events occurred.

[0070] The data sources involved in the multimedia materials include "XX Research", "XX Diary", and encyclopedia materials. The data information is shown in Table 1.

[0071] Table 1 Basic geographic data display

[0072]

[0073] S2: Perform HGIS digital processing on the acquired raw data;

[0074] S21: Perform HGIS digitization on historical maps, specifically:

[0075] Map preprocessing: includes map scanning, image enhancement, and distortion correction to improve map quality and readability.

[0076] Map registration: Convert the non-standard geographic coordinates on historical maps into the standard coordinate system used in modern GIS systems, and unify the scales of maps at different scales.

[0077] Verification between the past and present: Use reference maps or existing geographic data to correct errors in historical maps. Ensure data accuracy and completeness through overlay analysis and error detection, specifically:

[0078] 1. Digitization of historical data: Scan paper historical maps (aerial topographic maps) into digital images with high precision and perform vector extraction using GIS software; extract location descriptions (such as street names, building names), event coordinates or hand-drawn schematics from text materials and convert them into geographic reference information.

[0079] 2. Acquisition of modern geographic data: Obtain Bing Map modern maps and administrative division data as basic geographic data, and unify all data into the same coordinate system.

[0080] 3. Overlay analysis: Overlay the historical map vector layer with the modern map, select the same-name features (city gates, bridges, etc.) in the two types of maps as control points (GCPs), and mark the coordinate differences.

[0081] 4. Geometric correction: The annotated points with the same name are input as control points. After selecting the polynomial model to resample the historical map, the root mean square error is calculated for accuracy verification. It is then iteratively optimized by reselecting control points, and the aligned vector data is output.

[0082] 5. Attribute correction: Correct old place names in historical maps based on encyclopedia information, and adjust missing or ambiguous area labels in historical maps based on the locations of events described in historical documents.

[0083] Map vectorization: Converting features (points, lines, and areas) on a paper map into digital format, including manual and automatic digitization techniques, specifically:

[0084] 1. Manual digitization: With the help of GIS software, create corresponding feature layers according to the map feature type, create point, line, and surface layers, and set a suitable coordinate system for the newly created layers to ensure consistency with the geographic coordinates of the map; set the attributes of the digitized features for each layer; after digitization is completed, save the results as a Shapefile data file and check whether any features are missed and whether the coordinates are accurate.

[0085] 2. Automatic digitization: Use a scanner to scan paper maps into raster images, generate digital image files, and perform pre-processing operations such as denoising, contrast enhancement, and brightness adjustment on the scanned images; perform geospatial registration and correction on the raster images through control points, and convert the images to a coordinate system consistent with the actual geographic coordinates; select specific graphics and line colors in the raster images as targets, and rasterize the extracted and saved information; establish topological relationships for the generated vector files, and fully verify them with reference to the original paper maps.

[0086] Attribute Assignment: Add attribute information to digitized geometric objects, such as age, origin, and geographic description. Use a database management system to maintain this attribute data, including the selection of data models and the maintenance of data associations.

[0087] This paper uses the HGIS method to digitize old maps; uses historical text OCR technology to organize and form a thematic geographic database; and collects multiple types of data to prepare for narrative.

[0088] S22: Perform HGIS digital processing on historical documents, specifically:

[0089] Text digitization: Digitize documents, using scanning equipment and optical character recognition (OCR) technology to convert paper documents into electronic text. Clean the OCR-converted text, including correcting recognition errors, removing irrelevant symbols, and adjusting formatting.

[0090] Keyword extraction: This involves performing word segmentation, especially for Chinese text, to ensure that the text is correctly segmented for better semantic analysis. Natural language processing techniques, such as named entity recognition (NER), are applied to automatically identify and extract key information from the text, including time, place, people, and events. Part-of-speech tagging is used to assist in identification based on the context, ensuring that the extracted keywords accurately represent their role and meaning in the text.

[0091] Spatialization of geographic information: For the extracted location keywords, perform geocoding, coordinate conversion, map positioning and other operations through historical data and current geographic information queries.

[0092] Attribute association: Associate time, people, events and other keywords with the located geographic data.

[0093] S23: Perform HGIS digital processing on multimedia data, specifically:

[0094] Data collection: Collect data in the form of text, audio, 3D models, images, and pictures.

[0095] Format Conversion: Convert unstructured text data into a structured format for database storage or GIS attribute association. Convert different formats into common plain text for keyword extraction. Geotag audio, video, and image files to associate them with spatial coordinates in HGIS. Convert GIS data (e.g., Shapefiles) into formats supported by AR engines (e.g., 3D Tiles, GLTF / GLB), including 3D coordinates, outlines, and heights of buildings.

[0096] Data compression: Reduce the number of 3D model faces, compress image size, compress video frame rate, etc., transcode into a lightweight format supported by AR, reduce resource consumption, and adapt to mobile performance.

[0097] S3: Save the data processed by S2;

[0098] The historical maps digitized by HGIS are input into the basic geographic database; the historical document materials digitized by HGIS are input into the thematic geographic database; and the multimedia data digitized by HGIS are input into the multimedia material library.

[0099] S4: Build a historical narrative platform based on augmented reality technology to realize map narrative;

[0100] This paper constructs a historical narrative technology route and platform based on AR technology. First, the scene is designed in AR based on geographic data, which is developed based on Unity. Then, image recognition and matching are performed based on Python to realize the loading of AR three-dimensional models based on pictures. Based on this, different gestures of users on the screen are recognized to achieve different model interaction effects. Finally, all content is integrated into the mobile phone app "XXX" for management and update. The specific content is:

[0101] S41: Build scenarios based on geographic databases and geographic environment reproduction;

[0102] Reproduction of geographical environment: Load the basic environment configured in Mapbox in Unity, such as mountains, rivers, buildings, etc., add multi-layer geographical data, and design its layer organization and content.

[0103] Specifically, the data stored in S3 is stored in the Mapbox backend in the cloud, and the basic environment is configured in Mapbox Studio based on the geographic hierarchy. This allows the historical space corresponding to the research area to be presented. The creator can then read the geographic attributes in the cloud and use visual storytelling metaphors, combined with the main narrative line of the map, to construct the symbols and auxiliary elements of the map elements to form a basic planar geographic environment.

[0104] Finally, the geographic data is stylized in three dimensions, and multimedia materials such as pictures, voice, and video are introduced to bind the geographic objects to the multimedia materials.

[0105] S42: Present content in a set scene based on thematic geographic database, multimedia material library and image recognition;

[0106] Using image recognition technology, load the target image and feature template, and use OpenCV's ORB algorithm for feature extraction; ORB algorithm feature extraction includes three parts, specifically:

[0107] (1) In terms of spatial position, the FAST algorithm is used to detect corner points in the image (such as object edges and texture mutation areas). Specifically:

[0108] 1. Select candidate pixels: Traverse each pixel in the image and check whether it is a focus candidate; take the Bresenham circle with a radius of 3 pixels centered on the pixel, and a total of 16 equidistant circular pixels (numbered 1 to 16) to define its neighborhood range.

[0109] 2. Quick pre-screening: Set a brightness difference threshold and check the four points numbered 1, 5, 9, and 13 on the circumference, i.e., in the vertical and horizontal directions. If the absolute value of the brightness difference of at least three of the points satisfies the brightness difference greater than the threshold, the pixel is likely a corner point and proceeds to the next step. Otherwise, it is skipped directly.

[0110] 3. Continuous pixel detection: Find a continuous series of N points (usually N=9 or N=12) among the 16 points on the circumference. If the points meet one of the conditions of being brighter or darker than the center, the pixel can be determined as a corner point.

[0111] 4. Non-maximum suppression (NMS): eliminates duplicate corner points in dense areas and retains the most significant corner points.

[0112] (2) In terms of direction attributes, the intensity centroid method is used to calculate the main direction of the key point, the centroid of the neighborhood of the feature point is calculated by the geometric moment formula, and the feature point and the centroid are connected to form the main direction (range 0-360 degrees); the geometric moment formula is calculated as follows:

[0113] ,

[0114] Among them, I(x,y) is the grayscale value of the pixel in the image at point (x,y), The coordinates of the center of mass are, Represents the geometric moment, reflecting the spatial distribution characteristics of pixel intensity I(x,y) in the neighborhood; 、 Represents the power operation of pixel coordinates x and y. p and q refer to the order of the geometric moment (non-negative integers). The value determines the order and direction of the moment. For example, the first-order moment (p+q=1) describes the linear distribution of pixel intensity. Indicates the azimuth of the centroid relative to the feature point; Represents the component of the first-order moment in the x direction, that is, the sum of the x coordinates of all pixels multiplied by their intensities; Represents the component of the first-order moment in the y direction, that is, the sum of the y coordinates of all pixels multiplied by their intensities; represents the zero-order moment, which is the sum of all pixel intensities in the neighborhood; Represents the four-quadrant inverse tangent function.

[0115] (2) In terms of scale information, multi-scale detection is achieved through image pyramid (the default is 8-layer pyramid, scaling factor 1.2), specifically:

[0116] 1. Construct a Gaussian pyramid: Build a pyramid layer by layer, starting with the original image at layer 0. Apply a Gaussian filter to the image at layer k-1 (k = 1, 2, ..., 7), reducing the size of the blurred image to 1 / 1.2 times, to obtain the image at layer k.

[0117] 2. Multi-scale target detection: Use the FAST algorithm to extract feature points and combine them with descriptors for matching.

[0118] 3. Mapping and fusion of detection results: Map the coordinates detected at each layer to the position in the original image, and scale the width and height of the detection frame accordingly. Sort all detection frames by confidence, retain the frame with the highest confidence, and remove other frames whose IoU (intersection over union) exceeds the threshold. Repeat until all frames are processed.

[0119] Brute Force (BF) matching is performed on features. Regarding the brute force matching rule based on ORB features in OpenCV, its core matching rule is the full comparison principle. The matching process is as follows:

[0120] (1) Distance metric: Using the modified Hamming distance, the number of bits of difference between binary descriptors is calculated through XOR operation. ORB descriptors are usually 256 bits, and the distance range is 0 (complete match) to 256 (complete mismatch).

[0121] (2) Nearest neighbor criterion: For each descriptor in the query set, traverse all descriptors in the training set and select the matching pair with the smallest distance as the candidate.

[0122] (3) Threshold filtering: Set a maximum distance threshold (such as an empirical value of 30), or use the nearest neighbor ratio method (NNDR), that is, the ratio of the optimal matching distance to the suboptimal distance must be less than 0.7-0.8 to exclude fuzzy matches.

[0123] (4) Cross-validation: Only the matching pairs that are the nearest neighbors of each other (i.e., A→B and B→A) are retained to avoid the situation where one pair is matched with multiple pairs.

[0124] Based on the correspondence between 3D points (model coordinate system) and 2D projection points (image coordinate system), the camera pose (rotation matrix R and translation vector t) is solved. The 3D model is loaded and positioned, and the solution is performed using PnP.

[0125] By binding the model to the scene center, the vertex coordinates of the 3D model are scaled to match the scale of the scene, ensuring that the model is aligned with the image after projection. Subsequently, the model center point (such as the center of mass) is bound to the image center, and the model position is adjusted through translation transformation. The specific calculation formula is:

[0126] ,

[0127] Among them, K is the camera internal parameter, is the coordinate of the image center; Represents the adjusted translation vector, which is used to translate the center of the model to the target position aligned with the center of gravity of the image; t represents the translation vector, which is a three-dimensional vector; project() represents the projection function, which is used to project the 3D point coordinates onto the 2D image plane; R represents the rotation matrix, which is a 3×3 orthogonal matrix that describes how to rotate the object from the local coordinate system to the world coordinate system; Indicates the center point of the projected model, indicating the position of the center of the 3D model in the image.

[0128] S43: Interactive settings and experience based on gesture recognition and content presentation;

[0129] Using gesture recognition technology, we can obtain screen gesture contacts, determine whether they intersect with interactive objects, and make the model respond to gestures.

[0130] First, the gesture data is preliminarily cleaned and converted. If it is determined that the gesture has indeed interacted with a specific UI element (i.e., "intersected"), the corresponding event or action is triggered according to the predetermined logic, and the response processing is implemented, providing feedback to the user through visual effects, sounds, etc. The key nodes are:

[0131] (1) Intersection detection loop: traversal detection of all touch points and UI elements is achieved through nested loops;

[0132] (2) Conflict handling logic: When multiple touchpoints interact with an element, the only valid interaction is filtered through priority strategies (time, space, type);

[0133] (3) Gesture type matching: Mapping to predefined gesture types (click / slide / zoom) based on characteristics such as direction and speed;

[0134] (4) Feedback mechanism: Visual highlights (such as element color change) are triggered simultaneously with sound prompts to enhance the user experience.

[0135] like Figure 2As shown in the figure, it is a pseudo-code logic diagram for gesture recognition. First, the screen touch data is obtained, and then the data is pre-processed to filter out invalid touch points; it is determined whether there are any valid touch points left; if so, all touch points and UI elements are traversed to detect whether each touch point intersects with the UI element. If not, the next touch point is checked. If the touch point intersects with the UI element, the touch point and element pair is recorded, and then all touch points and UI elements are traversed; it is checked whether there are any unchecked touch points. If so, it is returned to the previous step to continue the detection; if not, it is determined whether there are multiple interaction conflicts. If there are multiple interaction conflicts, the conflicts are sorted according to time, space and Sort by priority of the type, select the highest priority interaction from the sorted interactions for processing, extract gesture features related to the highest priority interaction, such as direction, speed, and trajectory; match gesture types; if there are no multiple interaction conflicts, directly extract gesture features: direction, speed, trajectory; match gesture types; based on the matching results, determine the gesture type, if the gesture type is click, trigger a click event, if the gesture type is slide, trigger a slide event, if the gesture type is zoom, trigger a zoom event; the trigger-based event system is based on user feedback, such as highlighting the interacted UI elements and playing sounds.

[0136] S44: Manage and update content based on changed assets, interactive settings, and experiences to create map narratives.

[0137] Manage and update relevant content such as character narrative, time narrative, space narrative, visual narrative, multi-line narrative, etc. The organization of each narrative is as follows: Figure 4 As shown, to ensure the completeness and accuracy of the map story.

[0138] Users can create historical maps based on the app, or experience existing works. Both creation and experience are based on the narrative of historical maps.

[0139] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A historical map narrative method based on augmented reality, characterized by: The following steps are involved: S1: Get raw data; S2: Perform HGIS digital processing on the acquired raw data; S3: Save the data processed by S2; S4: Build a historical narrative platform based on augmented reality technology to realize map narrative, specifically: S41: Reproduce planning scenarios based on geographic databases and geographical environments; S42: Present content in planned scenarios based on thematic geographic databases, multimedia libraries, and image recognition; Using image recognition technology, load the target image and feature template, use OpenCV's ORB algorithm to extract features, perform brute force matching on the features, load the 3D model and locate it, and bind the model to the center of the image; ORB algorithm performs feature extraction including: In terms of spatial position, the FAST algorithm is used to detect corner points in the image; In terms of direction attributes, the intensity centroid method is used to calculate the main direction of the key point, and the centroid of the feature point neighborhood is calculated using the geometric moment formula. The feature point and the centroid are connected to form the main direction. The geometric moment formula is calculated as follows: m pq =∑ x,y x p y q I(x,y),θ=arctan2(m 01 ,m 10 ), Among them, I(x,y) represents the pixel gray value of the image at point (x,y), and the centroid coordinates are m pq represents the geometric moment; x p 、y q Represents the power operation of pixel coordinates x and y, p and q refer to the order of the geometric moment; θ represents the azimuth angle of the center of mass relative to the feature point; m 01 Represents the component of the first-order moment in the x direction, m 10 Represents the component of the first-order moment in the y direction; m 00 represents the zero-order moment, arctan2 represents the four-quadrant inverse tangent function; In terms of scale information, multi-scale detection is achieved through image pyramid; Solve the camera pose based on the correspondence between the model coordinate system and the image coordinate system; By binding the model to the scene center, the vertex coordinates of the 3D model are scaled to match the scale of the scene, ensuring that the model is aligned with the image after projection. Subsequently, the center point of the model is bound to the center of the image, and the model position is adjusted through translation transformation. The specific calculation formula is: t adjusted =t+K -1 ·(u center -project(R·p model-center +t)), Among them, K represents the camera internal parameter, u center represents the center coordinate of the image; t adjusted Represents the adjusted translation vector; t represents the translation vector; project() represents the projection function; R represents the rotation matrix, p model-center Indicates the center point of the projected model; S43: Interactive settings and experience based on gesture recognition and content presentation; S44: Manage and update content based on changed assets, interactive settings, and experiences to create map narratives.

2. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S1, multi-source data collection is carried out, including historical maps, historical documents and multimedia data.

3. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S2, the specific process of HGIS digitization of the acquired raw data is as follows: S21: Perform HGIS digitization of historical maps, including: Map preprocessing, map registration, ancient and modern verification, map vectorization and attribute assignment; S22: Perform HGIS digitization of historical documents, including: Text digitization, keyword extraction, geographic information spatialization and attribute association; S23: Perform HGIS digital processing on multimedia data, including: Data collection, format conversion and data compression.

4. The method for historical map narrative based on augmented reality according to claim 1, characterized in that: In S3, data storage: The specific contents of saving the data processed by S2 are: The historical maps digitized by HGIS are input into the basic geographic database; the historical document data digitized by HGIS are input into the thematic geographic database; and the multimedia data digitized by HGIS are input into the multimedia material library.

5. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In terms of spatial position, the specific content of detecting corner points in an image using the FAST algorithm is as follows: Select candidate pixels: traverse each pixel in the image to check whether it is a focus candidate; take the pixel as the center and select 16 equally spaced circular pixels with a radius of 3 pixels to define its neighborhood range; Quick pre-screening: Set a brightness difference threshold and check the four points numbered 1, 5, 9, and 13 on the circumference. If the absolute value of the brightness difference of three or more points is greater than the threshold, the pixel may be a corner point and proceed to the next step. Otherwise, skip directly. Continuous pixel detection: Find a continuous series of N points among the 16 points on the circumference that meet one of the conditions of being brighter or darker than the center, and determine that the pixel is a corner point; Non-maximum suppression: eliminates repeated corner points in dense areas and retains the most significant corner points.

6. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S42, the brute force matching rule based on ORB features has a core matching rule of full comparison principle. The matching process is as follows: Distance metric: Use the improved Hamming distance to calculate the number of bits of difference between binary descriptors through XOR operation; Nearest neighbor criterion: For each descriptor in the query set, traverse all descriptors in the training set and select the matching pair with the smallest distance as the candidate; Threshold filtering: Eliminate fuzzy matches by setting a maximum distance threshold or using the nearest neighbor ratio method; Cross-validation: Keep the matching pairs that are each other's nearest neighbors.

7. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S43, the gesture data is first preliminarily cleaned and converted. Gesture recognition technology is used to obtain the screen gesture contact points and determine whether they intersect with interactive objects. If so, the corresponding event or action is triggered according to the predetermined logic to implement response processing and provide feedback to the user through visual effects and sound. The key nodes are: Intersection detection loop: implement traversal detection of all contacts and UI elements through nested loops; Conflict handling logic: When multiple touchpoints interact with an element, the priority strategy is used to filter the only valid interaction; Gesture type matching: Mapping to predefined gesture types based on direction, speed, and trajectory characteristics; Feedback mechanism: Visual highlights and sound prompts are triggered synchronously to enhance user experience.

Images