Historical map narrative method based on augmented reality

Abstract

The invention discloses a historical map narrating method based on augmented reality, and belongs to the technical field of cartography and augmented reality. The method comprises the following steps: S1, acquiring original data; s2, carrying out HGIS digital processing on the obtained original data; s3, storing the data processed in the step S2; and S4, constructing a historical narrative platform based on an augmented reality technology, and realizing map narrative. The multimedia data and the digital map are embedded into the AR environment in an interactive mode for historical narration, and the defect that a traditional map is mainly applied to geographic data visualization and spatial analysis is overcome; according to the method, a transferable technical platform and a map narration template are provided for historical map narration, and the technical threshold of AR map compilation and historical memory and propagation is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical fields of cartography, narratology, historical GIS, and augmented reality, and specifically relates to a historical map narrative method based on augmented reality. Background Art

[0002] As a spatial visualization language, maps are the core tools for presenting geographic information and disseminating spatial knowledge, and they play an important role in reconstructing historical space and disseminating historical memory. Historical geographic information system (HGIS) can migrate traditional maps (old maps) and historical geographic data to geospatial information infrastructure with unified spatiotemporal geographic coordinates, and reproduce lost historical scenes in the form of digital maps. It is a digital platform for reproducing historical time and space and inheriting historical memory. Despite this, at present, most of the reproduction of maps by HGIS is still used for the spatialization and spatial analysis of historical information, and does not involve universal map production and cultural heritage dissemination. At present, map presentation technologies such as narrative maps, pan-maps, and scenario studies have not specifically considered the narrative under historical maps, and lack effective methods to express the history in historical maps.

[0003] Augmented Reality (AR) is a digital technology that integrates the real and the virtual. It is to superimpose digital content onto physical objects (such as paper maps) or locations in the real environment. AR technology has the characteristics of combining the real and the virtual, integrating online and offline resources, real-time interaction, and registration in 3D form. Key technologies include image recognition, plane recognition, SLAM and other methods. Based on image recognition, creating AR content on the map can effectively improve the map's performance dimension and expand the map's presentation and application boundaries.

[0004] The background technology of AR maps currently mainly involves the following aspects. First, in the process of map construction and data processing, map data collection and processing need to combine computer vision and image processing technology to perform data preprocessing, feature extraction and matching operations to provide basic data support for the construction of AR maps. However, it is necessary to solve the problem of fusing geospatial data from different sources and formats in order to fully and accurately construct an AR map database.

[0005] Secondly, the main technology involved in geographic information enhancement and visualization is augmented reality overlay technology. Based on the position and direction sensor data, virtual geographic information elements (such as annotations, icons, three-dimensional models, etc.) are accurately superimposed on the corresponding positions in the real world to enhance and enrich the real scene. AR provides a new method for geospatial data visualization, that is, a way to integrate virtual information into the real scene, such as the AR expression framework of locatable video objects in the existing technology, the projection-type augmented map system, and the research framework of AR visualization system. However, the core issue of augmented reality visualization is the fusion effect of the visualized object with the real world and the rationality of abstract information. Current research still pursues realistic effects and diversified expressions from the perspective of visualization, and there is relatively little research on the rationality of abstract information. Therefore, AR visualization lacks purpose and pertinence.

[0006] The current augmented reality map narrative combines virtual information and real scenes, but its technical implementation and practical application still face many challenges. For example, in terms of user experience, there may be information overload (too many virtual tags blocking the line of sight), unnatural interaction (gesture / voice recognition delay), dizziness caused by long-term use, etc. At the same time, different users have different acceptance and understanding of the complexity of the AR interface. Historical map narrative based on augmented reality (AR) is a further development of visualization tools that combine spatial data and storytelling. The core goal is to guide users to immerse themselves in a specific narrative logic through maps, but it is difficult to take into account how to naturally combine linear or non-linear storylines with geographic spatial information, balance information density and narrative rhythm, resource loading delays or format conflicts when integrating multimedia content, and narrative universality in multicultural contexts. In addition, AR maps are not yet popular among the public at this stage, and their functions are limited to route navigation. There are almost no AR map products for historical narratives, and there is also a lack of a universal technical route for historical map narratives based on AR. Summary of the invention

[0007] In response to the problems mentioned in the background technology, the present invention proposes a historical map narrative method based on augmented reality (AR), applies HGIS technology to universal map production, and adds AR technology. Through this method, a universal framework is provided for the production of AR maps, which lowers the production threshold of historical map narratives and breaks through the communication threshold between cartographers and map users.

[0008] Technical solution: In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: A historical map narrative method based on augmented reality includes the following steps: S1: Get raw data; S2: Perform HGIS digital processing on the acquired raw data; S3: save the data processed by S2; S4: Build a historical narrative platform based on augmented reality technology to realize map narrative, specifically: S41: Reproduce planning scenarios based on geographic databases and geographic environments; S42: Present content in planned scenarios based on thematic geographic databases, multimedia material libraries, and image recognition; S43: Interactive settings and experience based on gesture recognition and content presentation; S44: Manage and update content based on changed assets and interactive settings and experiences to create map narratives.

[0009] Preferably, in S1, multi-source data collection is performed, including historical maps, historical documents and multimedia data.

[0010] Preferably, in S2, the specific process of performing HGIS digital processing on the acquired raw data is as follows: S21: Perform HGIS digitization of historical maps, including: Map preprocessing, map registration, ancient and modern verification, map vectorization and attribute assignment; S22: Perform HGIS digital processing on historical documents, including: Text digitization, keyword extraction, geographic information spatialization and attribute association; S23: Perform HGIS digital processing on multimedia data, including: Data collection, format conversion and data compression.

[0011] Preferably, in S3, data storage: the specific contents of saving the data processed by S2 are: The historical maps digitized by HGIS are input into the basic geographic database; the historical document data digitized by HGIS are input into the thematic geographic database; and the multimedia data digitized by HGIS are input into the multimedia material library.

[0012] Preferably, in S42, image recognition technology is used to load the target image and feature template, OpenCV's ORB algorithm is used to extract features, brute force matching is performed on the features, the 3D model is loaded and positioned, and the center of the model and the image is bound.

[0013] Preferably, in S42, the feature extraction performed by the ORB algorithm includes: In terms of spatial position, the FAST algorithm is used to detect corner points in the image; In terms of direction attributes, the intensity centroid method is used to calculate the main direction of the key point, the centroid of the neighborhood of the feature point is calculated by the geometric moment formula, and the feature point and the centroid are connected to form the main direction. The geometric moment formula is calculated as follows: , Among them, I(x,y) represents the pixel gray value of the image at point (x,y), and the centroid coordinates are , represents geometric moment; , represents the power operation of pixel coordinates x and y, where p and q refer to the order of the geometric moment; Indicates the azimuth of the centroid relative to the feature point; represents the component of the first-order moment in the x direction, represents the component of the first-order moment in the y direction; represents the zero-order moment, represents the four-quadrant inverse tangent function; In terms of scale information, multi-scale detection is achieved through image pyramid.

[0014] Preferably, in terms of spatial position, the specific contents of detecting corner points in an image by using the FAST algorithm are as follows: Select candidate pixels: traverse each pixel in the image to check whether it is a focus candidate; take the pixel as the center and select 16 equidistant circular pixels with a radius of 3 pixels to define its neighborhood range; Quick pre-screening: Set the brightness difference threshold and check the four points numbered 1, 5, 9, and 13 on the circumference. If the brightness of more than three points satisfies the absolute value of the brightness difference greater than the threshold, the pixel may be a corner point and proceed to the next step. Otherwise, skip directly. Continuous pixel detection: Find a continuous series of N points among the 16 points on the circumference that meet one of the conditions of being brighter or darker than the center, and determine that the pixel is a corner point; Non-maximum suppression: eliminates repeated corner points in dense areas and retains the most significant corner points.

[0015] As a preferred embodiment, in S42, the core matching rule of the brute force matching rule based on the ORB feature is the principle of full comparison, and the matching process is specifically as follows: Distance metric: Improved Hamming distance is used to calculate the number of bits of difference between binary descriptors through XOR operation; Nearest neighbor criterion: For each descriptor in the query set, traverse all descriptors in the training set and select the matching pair with the smallest distance as the candidate; Threshold filtering: Eliminate fuzzy matches by setting a maximum distance threshold or using the nearest neighbor ratio method; Cross-validation: Keep the matching pairs that are each other's nearest neighbors.

[0016] Preferably, in S42, the camera pose is solved according to the correspondence between the model coordinate system and the image coordinate system; By binding the model to the center of the scene, the vertex coordinates of the 3D model are scaled to match the scale of the scene to ensure that the model is aligned with the image after projection; then, the center point of the model is bound to the center of the image, and the model position is adjusted through translation transformation. The specific calculation formula is: , Among them, K represents the camera internal parameter, Indicates the coordinates of the center of the image; represents the adjusted translation vector; t represents the translation vector; project() represents the projection function; R represents the rotation matrix, Indicates the center point of the projected model.

[0017] Preferably, in S43, the gesture data is firstly cleaned and converted, and the gesture recognition technology is used to obtain the screen gesture contact point to determine whether it intersects with the interactive object. If so, the corresponding event or action is triggered according to the predetermined logic to implement the response processing, and the user is provided with feedback of successful operation through visual effects and sounds. The key nodes are: Intersection detection loop: implement traversal detection of all touch points and UI elements through nested loops; Conflict handling logic: When multiple touchpoints interact with an element, the only valid interaction is filtered through the priority strategy; Gesture type matching: Mapping to predefined gesture types based on direction, speed and trajectory features; Feedback mechanism: Visual highlights and sound prompts are triggered synchronously to enhance user experience.

[0018] Beneficial effects: Compared with the prior art, the present invention has the following advantages: (1) For the first time, multimedia data (video, audio, web pages, text, etc.) and digital maps are embedded in an interactive form into the AR environment for historical narrative, making up for the shortcomings of traditional maps that are mainly used for geographic data visualization and spatial analysis.

[0019] (2) In the present invention, AR technology is combined to explore a new method of historical map narrative, and the environment setting, image recognition, and gesture recognition logic of the AR platform are encapsulated and adapted. The technical solution of the present invention has stronger narrative and communication properties than the current two-dimensional solution based on the website.

[0020] (3) The present invention provides a portable technical platform and map narrative template for historical map narratives, while supporting real-time interaction between cartographers and users, lowering the technical threshold for AR map compilation and historical memory dissemination, and increasing the speed of product updates and iterations.

[0021] (4) The feature extraction and matching implemented by the present invention based on the ORB algorithm can achieve more than 80% accurate matching and recognition in actual product scenarios (scores under the Vuforia platform), and realize accurate model positioning under user movement and different gestures, and is smooth and unobstructed in actual applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the historical map narrative method based on augmented reality of the present invention; Figure 2 It is the pseudo code logic diagram of gesture recognition of the present invention; Figure 3 is a schematic diagram of an area that can be scanned in the practice of the present invention, wherein (a) is a front map, and (b) is a back map; Figure 4 It is a schematic diagram of organizing the narrative of a geographic map according to the present invention; Figure 5 It is a schematic diagram of the model matching and recognition accuracy optimized based on the ORB algorithm in the present invention (the verification platform is Vuforia). DETAILED DESCRIPTION

[0023] The present invention is further illustrated below in conjunction with specific examples. The examples are implemented based on the technical solutions of the present invention. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0024] The historical map narrative method based on augmented reality provided in this embodiment mainly includes the following steps: S1: Get raw data; Collect historical maps, historical documents and multimedia data.

[0025] The data sources of historical maps include 1:10000 aerial survey topographic maps of a certain area in a certain historical period, "Architectural Maps of a Certain City in a Certain Historical Period", and Bing Map, covering relevant content such as the places where historical events took place, former residences of celebrities, and contemporary buildings.

[0026] The data sources of historical documents include "Diary of Someone", which covers specific content such as the time when historical events occurred.

[0027] The data sources involved in the multimedia materials include "XX Research", "XX Diary", and encyclopedia materials. The data information is shown in Table 1.

[0028] Table 1 Basic geographic data display

[0029] S2: Perform HGIS digital processing on the acquired raw data; S21: Perform HGIS digitization on historical maps, specifically: Map preprocessing: includes map scanning, image enhancement, and distortion correction to improve map quality and readability.

[0030] Map registration: Convert the non-standard geographic coordinates on historical maps into the standard coordinate system used in modern GIS systems, and unify the scales of maps at different scales.

[0031] Ancient and modern verification: Use reference maps or existing geographic data to correct errors on historical maps. Ensure data accuracy and completeness through overlay analysis and error detection, specifically: 1. Digitization of historical data: Scan paper historical maps (aerial topographic maps) into digital images with high precision, and use GIS software to extract vectorization; extract location descriptions (such as street names, building names), event coordinates or hand-drawn schematics from text materials and convert them into geographic reference information.

[0032] 2. Acquisition of modern geographic data: Obtain Bing Map modern maps and administrative division data as basic geographic data, and unify all data into the same coordinate system.

[0033] 3. Overlay analysis: Overlay the historical map vector layer with the modern map, select the same-name features (city gates, bridges, etc.) in the two types of maps as control points (GCPs), and mark the coordinate differences.

[0034] 4. Geometric correction: The annotated points of the same name are input as control points. After selecting the polynomial model to resample the historical map, the root mean square error is calculated for accuracy verification. It is iteratively optimized by reselecting control points, and then the aligned vector data is output.

[0035] 5. Attribute correction: Correct old place names in historical maps based on encyclopedia information, and adjust missing or blurred area labels in historical maps based on the location of events described in historical documents.

[0036] Map vectorization: Converting features (points, lines, and surfaces) on a paper map into digital format, including manual and automatic digitization techniques, specifically: 1. Manual digitization: With the help of GIS software, create corresponding feature layers according to the map feature types, create point, line, and surface layers, and set appropriate coordinate systems for the newly created layers to ensure consistency with the geographic coordinates of the map; set the attributes of the digitized features for each layer; after digitization is completed, save the results as a Shapefile data file, and check whether any features are missed and whether the coordinates are accurate.

[0037] 2. Automatic digitization: Use a scanner to scan paper maps into raster images, generate digital image files, and perform pre-processing operations such as denoising, contrast enhancement, and brightness adjustment on the scanned images; perform geospatial registration and correction on raster images through control points, and convert the images to a coordinate system consistent with the actual geographic coordinates; select specific graphics and line colors in the raster images as targets, and rasterize the extracted and saved information; establish topological relationships for the generated vector files, and fully verify them with reference to the original paper maps.

[0038] Attribute assignment: Add attribute information to digitized geometric objects, such as age, origin, and geographic description. Use a database management system to maintain these attribute data, including the selection of data models and the maintenance of data associations.

[0039] The present invention uses the HGIS method to digitize old maps; uses historical text OCR technology to sort out and form a thematic geographic database; and collects multiple types of data in preparation for narrative.

[0040] S22: Perform HGIS digital processing on historical documents, specifically: Text digitization: Digitize the data and convert paper documents into electronic text using scanning equipment and optical character recognition (OCR) technology. Clean the text after OCR conversion, including correcting recognition errors, removing irrelevant symbols and adjusting the format.

[0041] Keyword extraction: Perform word segmentation, especially for Chinese text, to ensure that the text can be correctly segmented for better semantic analysis, and apply natural language processing technology, such as named entity recognition (NER), to automatically identify and extract key information in the text, including time, place, people and events. Use part-of-speech tagging technology to assist in identification according to the context, to ensure that the role and meaning of the extracted keywords in the text are accurate.

[0042] Spatialization of geographic information: For the extracted location keywords, perform geocoding, coordinate conversion, map positioning and other operations through historical data and current geographic information queries.

[0043] Attribute association: associate time, people, events and other keywords with the located geographic data.

[0044] S23: Perform HGIS digital processing on multimedia data, specifically: Data collection: Collect data in the form of text, audio, 3D models, images, and pictures.

[0045] Format conversion: Convert unstructured text data into structured formats for database storage or GIS attribute association. Convert different formats into common plain text for keyword extraction. Embed geotags in audio, video, and image files and associate them with spatial coordinates in HGIS. Convert GIS data (such as Shapefile) into formats supported by AR engines (such as 3D Tiles, GLTF / GLB), including 3D coordinates, outlines, and heights of buildings.

[0046] Data compression: reduce the number of 3D model faces, compress image size, compress video frame rate, etc., transcode into a lightweight format supported by AR, reduce resource consumption, and adapt to mobile performance.

[0047] S3: save the data processed by S2; The historical maps digitized by HGIS are input into the basic geographic database; the historical document data digitized by HGIS are input into the thematic geographic database; and the multimedia data digitized by HGIS are input into the multimedia material library.

[0048] S4: Build a historical narrative platform based on augmented reality technology to realize map narrative; The present invention constructs a historical narrative technology route and platform based on AR technology. First, the scene is designed in AR based on geographic data, which is developed based on Unity. Then, image recognition and matching are performed based on Python to realize the loading of AR three-dimensional model based on pictures. Based on this, different gestures of users on the screen are recognized to achieve different model interaction effects. Finally, all content is integrated into the mobile phone APP "XXX" for management and update. The specific content is: S41: Building scenarios based on geographic databases and geographic environment reproduction; Reproduction of geographical environment: Load the basic environment configured in Mapbox such as mountains, rivers, buildings, etc. in Unity, add multi-layer geographical data, and design its layer organization and content.

[0049] Specifically, the data stored in S3 is stored in the cloud Mapbox backend, and the basic environment is configured in MapboxStudio based on the geographic location level, so that the historical space corresponding to the research area can be presented. After that, the producer can read the geographic attributes in the cloud, use visual narrative metaphors, combine the narrative main line of the map, construct the symbols and auxiliary elements of the map elements, and form a plane basic geographical environment.

[0050] Finally, the geographic data is stylized in three dimensions, and multimedia materials such as pictures, audio, and video are introduced to bind the geographic objects to the multimedia materials.

[0051] S42: Present content in the set scene based on thematic geographic database, multimedia material library and image recognition; Using image recognition technology, load the target image and feature template, and use OpenCV's ORB algorithm for feature extraction; ORB algorithm feature extraction includes three parts, specifically: (1) In terms of spatial position, the FAST algorithm is used to detect corner points in the image (such as object edges and texture mutation areas). Specifically: 1. Select candidate pixels: traverse each pixel in the image to check whether it is a focus candidate; take the Bresenham Circle with a radius of 3 pixels centered on the pixel, a total of 16 equidistant circular pixels (numbered 1 to 16) to define its neighborhood range.

[0052] 2. Quick pre-screening: Set the brightness difference threshold and check the four points numbered 1, 5, 9, and 13 on the circumference, that is, in the vertical and horizontal directions. If the brightness of at least three of the points satisfies the absolute value of the brightness difference greater than the threshold, the pixel may be a corner point and proceed to the next step, otherwise skip it directly.

[0053] 3. Continuous pixel detection: Find a continuous series of N points (usually N=9 or N=12) among the 16 points on the circumference. If the pixel satisfies one of the conditions of being brighter or darker than the center, the pixel can be determined to be a corner point.

[0054] 4. Non-maximum suppression (NMS): eliminates repeated corner points in dense areas and retains the most significant corner points.

[0055] (2) In terms of directional attributes, the intensity centroid method is used to calculate the main direction of the key point, the centroid of the neighborhood of the feature point is calculated by the geometric moment formula, and the feature point and the centroid are connected to form the main direction (range 0-360 degrees); the geometric moment formula is calculated as follows: , Among them, I(x,y) is the pixel gray value of the image at point (x,y), The coordinates of the centroid are, Represents the geometric moment, which reflects the spatial distribution characteristics of pixel intensity I(x,y) in the neighborhood; , It represents the power operation of pixel coordinates x and y. p and q refer to the order of geometric moments (non-negative integers). The value determines the order and directional characteristics of the moment. For example, the first-order moment (p+q=1) describes the linear distribution of pixel intensity. Indicates the azimuth of the centroid relative to the feature point; Represents the component of the first-order moment in the x direction, that is, the sum of the x coordinates of all pixels multiplied by their intensity; Represents the component of the first-order moment in the y direction, that is, the sum of the y coordinates of all pixels multiplied by their intensities; represents the zero-order moment, which is the sum of all pixel intensities in the neighborhood; Represents the four-quadrant inverse tangent function.

[0056] (2) In terms of scale information, multi-scale detection is achieved through image pyramid (the default is 8-layer pyramid, and the scaling factor is 1.2), specifically: 1. Construct a Gaussian pyramid: Take the original image as the 0th layer and generate the pyramid layer by layer. Apply Gaussian filtering to the k-1th layer (k=1, 2, ..., 7) image, reduce the size of the blurred image to 1 / 1.2 times, and obtain the kth layer image.

[0057] 2. Multi-scale target detection: Use the FAST algorithm to extract feature points and combine them with descriptors for matching.

[0058] 3. Mapping and fusion of detection results: Map the coordinates detected at each layer to the position in the original image, and scale the width and height of the detection frame accordingly. Sort all detection frames by confidence, retain the frame with the highest confidence, and remove other frames whose IoU (intersection over union) exceeds the threshold. Repeat until all frames are processed.

[0059] Brute Force (BF) matching is performed on features. Regarding the brute force matching rules based on ORB features in OpenCV, its core matching rule is the principle of full comparison. The matching process is as follows: (1) Distance metric: Improved Hamming distance is used to calculate the number of bits of difference between binary descriptors through XOR operation. ORB descriptors are usually 256 bits, and the distance range is 0 (complete match) to 256 (complete mismatch).

[0060] (2) Nearest neighbor criterion: For each descriptor in the query set, traverse all descriptors in the training set and select the matching pair with the shortest distance as the candidate.

[0061] (3) Threshold filtering: Set a maximum distance threshold (such as an empirical value of 30), or use the nearest neighbor ratio method (NNDR), that is, the ratio of the optimal matching distance to the suboptimal distance must be less than 0.7-0.8 to exclude fuzzy matches.

[0062] (4) Cross-validation: Only the matching pairs that are the nearest neighbors to each other (i.e., A→B and B→A) are retained to avoid the situation where one pair is matched with multiple pairs.

[0063] According to the correspondence between 3D points (model coordinate system) and 2D projection points (image coordinate system), solve the camera pose (rotation matrix R and translation vector t). Load the 3D model and locate it, and use PnP to solve it.

[0064] By binding the model to the center of the scene, the vertex coordinates of the 3D model are scaled to a scale that matches the scene, ensuring that the model is aligned with the image after projection; then, the center point of the model (such as the center of mass) is bound to the center of the image, and the model position is adjusted through translation transformation. The specific calculation formula is: , Among them, K is the camera internal parameter, is the image center coordinate; Represents the adjusted translation vector, which is used to translate the center of the model to the target position aligned with the center of gravity of the image; t represents the translation vector, which is a three-dimensional vector; project() represents the projection function, which is used to project the 3D point coordinates to the 2D image plane; R represents the rotation matrix, which is a 3×3 orthogonal matrix that describes how to rotate the object from the local coordinate system to the world coordinate system; Indicates the center point of the projected model, indicating the position of the center of the 3D model in the image.

[0065] S43: Interactive settings and experience based on gesture recognition and content presentation; Using gesture recognition technology, we can obtain screen gesture contacts, determine whether they intersect with interactive objects, and make the model respond to gestures.

[0066] First, the gesture data is preliminarily cleaned and converted. If it is determined that the gesture has indeed interacted with a specific UI element (i.e., "intersected"), the corresponding event or action is triggered according to the predetermined logic, and the response processing is implemented to provide the user with feedback on the success of the operation through visual effects, sounds, etc. The key nodes are: (1) Intersection detection loop: implement traversal detection of all touch points and UI elements through nested loops; (2) Conflict handling logic: When multiple touchpoints interact with an element, the only valid interaction is screened through priority strategies (time, space, type); (3) Gesture type matching: Mapping to predefined gesture types (click / slide / zoom) based on features such as direction and speed; (4) Feedback mechanism: Visual highlights (such as element color change) and sound prompts are triggered synchronously to enhance the user experience.

[0067] like Figure 2As shown in the figure, it is a pseudo code logic diagram of gesture recognition. First, the screen touch data is obtained, and then the data is preprocessed to filter out invalid touch points; it is determined whether there are any valid touch points left; if so, all touch points and UI elements are traversed to detect whether each touch point intersects with the UI element. If not, the next touch point is checked. If the touch point intersects with the UI element, the touch point and element pair are recorded, and then all touch points and UI elements are traversed; it is checked whether there are any unchecked touch points. If so, return to the previous step to continue the detection; if not, it is determined whether there are multiple interaction conflicts. If there are multiple interaction conflicts, the time, space and Sort the priorities of the types, select the highest priority interaction from the sorted interactions for processing, extract the gesture features related to the highest priority interaction, such as direction, speed, and trajectory; match the gesture type; if there are no multiple interaction conflicts, directly extract the gesture features: direction, speed, trajectory; match the gesture type; based on the matching results, determine the gesture type, if the gesture type is click, trigger a click event, if the gesture type is slide, trigger a slide event, and if the gesture type is zoom, trigger a zoom event; the triggered event system is based on user feedback, such as highlighting the interacted UI elements and playing sounds.

[0068] S44: Manage and update content based on changed assets and interactive settings and experiences to create map narratives; Manage and update the relevant contents such as character narrative, time narrative, space narrative, visual narrative, multi-line narrative, etc. The organization of each narrative is as follows: Figure 4 As shown, to ensure the completeness and accuracy of the map story.

[0069] Users can create historical maps based on the APP, or experience existing works. Both creation and experience are based on the narrative of historical maps.

[0070] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A historical map narrative method based on augmented reality, characterized by: The following steps are involved: S1: Get raw data; S2: Perform HGIS digital processing on the acquired raw data; S3: save the data processed by S2; S4: Build a historical narrative platform based on augmented reality technology to realize map narrative, specifically: S41: Reproduce planning scenarios based on geographic databases and geographic environments; S42: Present content in planned scenarios based on thematic geographic databases, multimedia material libraries, and image recognition; S43: Interactive settings and experience based on gesture recognition and content presentation; S44: Manage and update content based on changed assets and interactive settings and experiences to create map narratives.

2. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S1, multi-source data collection is carried out, including historical maps, historical documents and multimedia data.

3. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S2, the specific process of HGIS digitization of the acquired raw data is as follows: S21: Perform HGIS digitization of historical maps, including: Map preprocessing, map registration, ancient and modern verification, map vectorization and attribute assignment; S22: Perform HGIS digital processing on historical documents, including: Text digitization, keyword extraction, geographic information spatialization and attribute association; S23: Perform HGIS digital processing on multimedia data, including: Data collection, format conversion and data compression.

4. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S3, data storage: The specific contents of saving the data processed by S2 are: The historical maps digitized by HGIS are input into the basic geographic database; the historical document data digitized by HGIS are input into the thematic geographic database; and the multimedia data digitized by HGIS are input into the multimedia material library.

5. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S42, image recognition technology is used to load the target image and feature template, OpenCV's ORB algorithm is used for feature extraction, brute force matching is performed on the features, the 3D model is loaded and positioned, and the center of the model and the image are bound.

6. The historical map narrative method based on augmented reality according to claim 5, characterized in that: In S42, the feature extraction performed by the ORB algorithm includes: In terms of spatial position, the FAST algorithm is used to detect corner points in the image; In terms of direction attributes, the intensity centroid method is used to calculate the main direction of the key point, the centroid of the neighborhood of the feature point is calculated by the geometric moment formula, and the feature point and the centroid are connected to form the main direction. The geometric moment formula is calculated as follows: ， Among them, I(x,y) represents the pixel gray value of the image at point (x,y), and the centroid coordinates are , represents geometric moment; , represents the power operation of pixel coordinates x and y, where p and q refer to the order of the geometric moment; Indicates the azimuth of the centroid relative to the feature point; represents the component of the first-order moment in the x direction, represents the component of the first-order moment in the y direction; represents the zero-order moment, represents the four-quadrant inverse tangent function; In terms of scale information, multi-scale detection is achieved through image pyramid.

7. The historical map narrative method based on augmented reality according to claim 6, characterized in that: In terms of spatial position, the specific content of detecting corner points in an image using the FAST algorithm is as follows: Select candidate pixels: traverse each pixel in the image to check whether it is a focus candidate; take the pixel as the center and select 16 equidistant circular pixels with a radius of 3 pixels to define its neighborhood range; Quick pre-screening: Set the brightness difference threshold and check the four points numbered 1, 5, 9, and 13 on the circumference. If the brightness of more than three points satisfies the absolute value of the brightness difference greater than the threshold, the pixel may be a corner point and proceed to the next step. Otherwise, skip directly. Continuous pixel detection: Find a continuous series of N points among the 16 points on the circumference that meet one of the conditions of being brighter or darker than the center, and determine that the pixel is a corner point; Non-maximum suppression: eliminates repeated corner points in dense areas and retains the most significant corner points.

8. The historical map narrative method based on augmented reality according to claim 5, characterized in that: In S42, the core matching rule of the brute force matching rule based on the ORB feature is the principle of full comparison. The specific matching process is as follows: Distance metric: Improved Hamming distance is used to calculate the number of bits of difference between binary descriptors through XOR operation; Nearest neighbor criterion: For each descriptor in the query set, traverse all descriptors in the training set and select the matching pair with the smallest distance as the candidate; Threshold filtering: Eliminate fuzzy matches by setting a maximum distance threshold or using the nearest neighbor ratio method; Cross-validation: Keep the matching pairs that are each other's nearest neighbors.

9. The historical map narrative method based on augmented reality according to claim 5, characterized in that: In S42, the camera position and posture are solved according to the correspondence between the model coordinate system and the image coordinate system; By binding the model to the center of the scene, the vertex coordinates of the 3D model are scaled to match the scale of the scene to ensure that the model is aligned with the image after projection; then, the center point of the model is bound to the center of the image, and the model position is adjusted through translation transformation. The specific calculation formula is: ， Among them, K represents the camera internal parameter, Indicates the coordinates of the center of the image; represents the adjusted translation vector; t represents the translation vector; represents the projection function; R represents the rotation matrix, Indicates the center point of the projected model.

10. The historical map narrative method based on augmented reality according to claim 1, characterized in that: In S43, the gesture data is firstly cleaned and converted, and the gesture recognition technology is used to obtain the screen gesture contact points to determine whether they intersect with the interactive object. If so, the corresponding event or action is triggered according to the predetermined logic to implement the response processing, and provide the user with feedback on the successful operation through visual effects and sounds. The key nodes are: Intersection detection loop: implement traversal detection of all touch points and UI elements through nested loops; Conflict handling logic: When multiple touchpoints interact with an element, the only valid interaction is filtered through the priority strategy; Gesture type matching: Mapping to predefined gesture types based on direction, speed and trajectory features; Feedback mechanism: Visual highlights and sound prompts are triggered synchronously to enhance user experience.

Images