Crowdsourcing digital information trace leaving system and method for augmented reality environment

Abstract

The invention discloses a crowdsourcing digital information trace leaving system and method for an augmented reality environment, and relates to the technical field of augmented reality, the system comprises a client device and a cloud server, and data interaction is carried out between the client device and the cloud server; wherein the client equipment comprises an environment sensing module, a positioning module and a digital nick processing module; the cloud server comprises a map index module, a digital nicking module, a management module and an operation calculation module; the environment sensing module senses the environment and its own position based on a sensor. Professional team acquisition is replaced by a crowdsourcing mode, construction and dynamic extension of a global high-precision AR space map are realized at extremely low cost, and the core pain point of industry high-cost mapping is solved; the four interaction media cover discrete information marking-spatio-temporal backtracking-3D creation-2D anchoring of a whole scene, deep binding of digital information and a physical space is achieved, unprecedented AR social immersion is provided, and a sustainable profit mode is provided for a platform.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and more specifically to a crowdsourced digital information tracking system and method for augmented reality environments. Background Technology

[0002] Augmented Reality (AR) technology overlays digital information onto physical space in real time, creating fusion-based interactive scenarios for users and demonstrating immense application potential in areas such as social networking, navigation, and commercial displays. Digital imprinting refers to digital content entities created by users in the AR environment that are strongly correlated with their physical spatial location, including but not limited to spatial information tags, spatiotemporal image portals, spatial vector graffiti, and surface-anchored planar drawings. However, current AR technologies for digital information imprinting and spatial interaction still have significant limitations. They struggle to meet the demands of large-scale, low-cost, and highly immersive user experiences. High-precision AR spatial maps typically require pre-collection by professional teams using expensive equipment and cannot be dynamically expanded or updated according to user needs. Interaction methods are also limited: interactions are mostly confined to viewing simple 3D models or text and image information, lacking comprehensive spatial information recording methods from 2D annotation to 3D creation. Furthermore, there is a lack of sustainable business models: existing applications struggle to effectively combine user creative value with the platform's economic system, failing to support a thriving UGC ecosystem.

[0003] In existing technologies, such as Google's ARCore Geospatial API, which relies on pre-set global 3D map data, users cannot dynamically expand the map coverage. Meta's Visual Positioning System supports visual positioning, but does not provide a mechanism for the persistence and socialization of user-generated content. In addition, existing AR interactions are mostly limited to simple 3D model placement or image recognition, lacking creative means that are deeply bound to physical space, such as spatiotemporal image retrospection and 3D vector graffiti. In response, we propose a crowdsourced digital information recording system and method for augmented reality environments. Summary of the Invention

[0004] To address the aforementioned technical problems, a crowdsourced digital information tracking system and method for augmented reality environments is provided, which solves the problems described above.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a crowdsourced digital information tracking system for augmented reality environments, characterized in that the system includes: a client device and a cloud server, with data interaction between the client device and the cloud server; wherein the client device includes an environmental perception module, a positioning module, and a digital tracking processing module; the cloud server includes a map indexing module, a digital tracking module, a management module, and an operation calculation module;

[0006] The environmental perception module uses sensors to sense the environment and its own position, and collects environmental data.

[0007] The positioning module processes environmental data locally in real time, builds and expands a spatial map of the environment, and then uploads it.

[0008] The digital marking processing module receives user creation instructions, generates multimedia content based on interactive media, adds location tags to the content based on the spatial map output by the positioning module, and packages and uploads it into the digital marking module.

[0009] The map indexing module is used to create and access indexes on the spatial map data uploaded by crowdsourcing.

[0010] The digital etching module is used to store multimedia content created by users based on media, along with the spatial coordinates and duration of the content.

[0011] Preferably, the interactive media within the digital marking processing module include triggerable spatial information tags, spatiotemporal image portals, spatial vector graffiti, and surface-anchored planar drawings;

[0012] Triggerable spatial information tags are created in response to user commands at specified spatial coordinates. Clicking them displays and allows input of multimedia content including text, audio, and video. The content is stored in the form of associated spatial coordinates.

[0013] The spatiotemporal image portal is used to automatically generate a fixed area centered on the current location in response to user commands, guiding the user to capture image content in that area; when the user's device enters the fixed area later, the system generates a spatiotemporal window in the AR view through image registration and overlay technology to display historical images of that area;

[0014] Spatial vector graffiti is used to respond to the user's 3D drawing actions, generating vector graphics based on three-dimensional spatial coordinates, supporting free 3D creation, and storing and rendering graphic data in the form of three-dimensional vector coordinates;

[0015] The surface-anchored planar drawing is used to respond to the user's specification of the diagonal points of the physical surface and define a two-dimensional drawing plane. After the user completes 2D drawing in this plane, the system stores the brush strokes in the form of projected two-dimensional coordinates and adjusts the display synchronously with the change of the physical surface's viewing angle.

[0016] Preferably, the specific steps for constructing and expanding the spatial map of the environment within the positioning module are as follows:

[0017] Map initialization: For areas not covered by the map, the system prompts the user to become a spatial explorer; after the user clicks to start scanning, the client collects environmental data through the environmental awareness module. When the scan volume reaches a preset threshold, an initial spatial data packet is generated and uploaded to the cloud server; the cloud server creates new block records in the spatial map database.

[0018] User positioning and content loading: When a user enters the area later, the client device downloads the corresponding block data from the cloud server, captures the image in real time through the camera and matches it with the visual features of the block data; if the match is successful, AR positioning is performed and the digital markings of the area are loaded simultaneously.

[0019] Dynamic map expansion: If a user enters an uncovered sub-area of ​​an existing map area, the client device automatically triggers map expansion, collects new area data, generates a map expansion package, and uploads it to the cloud server; the cloud merges the map expansion package with the original block data and updates the spatial map;

[0020] Cloud-based map stitching: When a user's movement trajectory connects two independent spatial blocks, the client device uploads the connectivity trajectory data; the cloud server triggers a map fusion algorithm to calculate the spatial transformation relationship between the two blocks and stitch them together into a unified, coherent large spatial map.

[0021] Preferably, the digital marking processing module receives touch, gesture, and voice and button creation commands from handheld devices and AR devices via a multimodal interface. It matches the corresponding interactive medium to call the sub-module, processes the high-frequency interaction between the spatial information tag's text, image, audio, and video, the spatiotemporal image portal's regional video, the trajectory vector graphics of vector graffiti and planar drawing, and the positioning module. Based on 6DoF pose data, it generates precise location tags: binding 3D coordinates and map feature points, generating a circular area with anchor points, and projecting planar coordinates; integrating the content and metadata into an encrypted data packet, uploading it to the cloud via HTTPS, and providing feedback to the user and caching it locally upon completion.

[0022] Preferably, the map index module interfaces with the map data package uploaded by the client, removes invalid data through format and quality checks, and performs deduplication preprocessing; the index construction adopts a composite structure of spatial hierarchy and metadata association; a three-level spatial index is built based on R tree, binding the three-dimensional boundaries of blocks and feature points; the metadata index is associated with block attributes, supporting multi-condition filtering.

[0023] Preferably, the system further includes: a management module, wherein the management module specifically includes:

[0024] Users can create public and private communities, and digital scratches are stored in categories by community ID. Client devices can switch communities via the top drop-down menu or a dedicated voice button, and the digital scratches of the current community are displayed after switching. Digital scratches are sorted by comprehensive score. The settings interface of the client device provides a slider and a number input box, allowing users to adjust the number of scratches displayed at one time, and provides a refresh button to update the content list.

[0025] Preferably, user commands include triggering via application interface buttons or via predefined air gestures; spatial vector graffiti shares basic drawing properties with surface-anchored planar drawing, while spatial vector graffiti has its own independent advanced brush tool library.

[0026] Preferably, the map fusion algorithm is as follows:

[0027] The process involves identifying independent blocks to be merged, along with associated trajectory and point cloud data; preprocessing to remove point cloud noise and filter feature points; finding cross-block feature association points through coarse matching using a fast nearest neighbor search library and fine matching using random sampling consistency; optimizing the spatial transformation model based on the iterative nearest point algorithm to unify the block coordinate system; verifying and handling geometric and semantic conflicts in the splicing region; using a sparse BA algorithm to reduce accumulated errors for multiple blocks; generating a unified map and updating the database on the cloud server; and employing an anomaly handling mechanism to address matching failures and error exceeding limits.

[0028] Preferably, the system also includes an operational computing module to provide system computing support, including executing map fusion algorithms, high-concurrency data processing, AI visual matching, and commercial billing; wherein high-concurrency data processing uses a hash consistency algorithm to distribute requests to 10-100 computing nodes; performs real-time decompression and format conversion on map data packets uploaded by the client, and performs preprocessing on digital marking content by segmenting and storing it; AI visual matching constructs a distributed feature index by synchronizing the core feature library of each block from the map index module; receives real-time environmental feature data uploaded by the client positioning module, and uses coarse matching and fine matching strategies to calculate feature overlap for matching; commercial billing includes personal billing, enterprise advertising billing, and billing risk control.

[0029] A crowdsourced digital information tracking method for augmented reality environments, comprising the following steps:

[0030] S1. After the client device is initialized, the environmental perception module collects environmental features and device pose data through sensors and transmits them to the positioning module.

[0031] S2. The positioning module builds an extended spatial map based on the environmental perception module and generates a data package to upload to the cloud map index module.

[0032] S3. When the user is creating content, the digital scratch processing module receives instructions to generate multimedia content, adds spatial tags by combining the data from the positioning module, and packages and uploads it to the cloud digital scratch module.

[0033] S4. The map index module verifies uploaded map data, creates indexes, and updates them dynamically.

[0034] S5, the digital marking module stores content and associates it with spatial information, and the management module filters markings by community ID;

[0035] S6, the operation and computing module performs map fusion, high-concurrency processing, AI visual matching and commercial billing, and finally the client completes the positioning, retrieves and renders the scratch content.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] This application utilizes a crowdsourcing model to replace professional teams for data collection, achieving the construction and dynamic expansion of a high-precision global AR spatial map at extremely low cost, thus addressing the core pain point of high-cost mapping in the industry. Four interactive media cover the entire scene, from discrete information labeling to spatiotemporal retrospection, 3D creation, and 2D anchoring, achieving a deep integration of digital information and physical space and providing an unprecedented AR social immersion experience. A layered business system directly links user creation, resource consumption, and platform revenue, incentivizing high-quality user creation and providing a sustainable profit model for the platform, supporting the prosperity of the UGC ecosystem. This forms a complete technological and commercial closed loop, with map data continuously enriched as the user base grows, and interactive and commercial functions attracting more users, building an ecosystem barrier that is difficult to replicate. Attached Figure Description

[0038] Figure 1 This is the system architecture and flowchart of the present invention;

[0039] Figure 2 This is a schematic diagram of the 2D output of the present invention;

[0040] Figure 3 This is a 2D input schematic diagram of the present invention;

[0041] Figure 4 This is a schematic diagram of the 3D output of the present invention;

[0042] Figure 5 This is a 3D input diagram of the present invention;

[0043] Figure 6 This is a schematic diagram of the image output of the present invention;

[0044] Figure 7 This is a schematic diagram of image input for the present invention;

[0045] Figure 8 This is a schematic diagram of the text output of the present invention;

[0046] Figure 9 This is a schematic diagram of text input for the present invention. Detailed Implementation

[0047] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0048] Reference Figure 1 As shown, a crowdsourced digital information tracking system for augmented reality environments is disclosed. The system includes: a client device and a cloud server, with data interaction between the client device and the cloud server; wherein the client device includes an environmental perception module, a positioning module, and a digital tracking processing module; the cloud server includes a map indexing module, a digital tracking module, a management module, and an operation computing module.

[0049] The environmental perception module uses sensors to sense the environment and its own position, and collects environmental data.

[0050] The positioning module processes environmental data locally in real time, builds and expands a spatial map of the environment, and then uploads it.

[0051] The digital marking processing module receives user creation instructions, generates multimedia content based on interactive media, adds location tags to the content based on the spatial map output by the positioning module, and packages and uploads it into the digital marking module.

[0052] The map indexing module is used to create and access indexes on the spatial map data uploaded by crowdsourcing.

[0053] The digital etching module is used to store multimedia content created by users based on media, and associates the spatial coordinates and duration of the content;

[0054] The management module is used to manage user account information and filter digital tick marks by community ID;

[0055] The operational computing module provides system computing support, including executing map fusion algorithms, high-concurrency data processing, AI visual matching, and business billing.

[0056] Traditional AR maps rely on professional teams and expensive equipment for data collection, resulting in high costs and fixed coverage. This system, however, uses a client-side positioning module to link user-sourced map creation. Ordinary users can build and expand spatial maps and upload them using device sensors, while the cloud-based map index module manages them uniformly, completely eliminating reliance on professional equipment. Simultaneously, the map expands in real-time as users explore, quickly covering multiple indoor and outdoor scenarios and resolving the issue of blank areas. The environmental perception module collects high-precision environmental data, the positioning module outputs accurate spatial maps, and the digital marking processing module uses this data to add centimeter-level spatial tags to multimedia content. Combined with AI visual matching from the cloud-based operational computing module, it ensures that digital content changes synchronously with the physical space perspective, without drift or jitter, achieving an immersive experience where digital information and physical space coexist, thus solving the pain point of traditional AR content's unstable anchoring.

[0057] In 2D / 3D rendering, it offers paid features for uploading images / 3D models, paid expansion of virtual object display space, and paid purchase of longer content duration, along with a content review mechanism that requires prior review before release; it provides a dedicated advertising mark type, supports displaying high-definition videos and animated 3D models in mid-air, and adopts an independent commercial billing and operation strategy.

[0058] The interactive media within the digital marking processing module include triggerable spatial information tags, spatiotemporal image portals, spatial vector graffiti, and surface-anchored planar drawings;

[0059] Triggerable spatial information tags are created in response to user commands at specified spatial coordinates. Clicking them displays and allows input of multimedia content including text, audio, and video. The content is stored in the form of associated spatial coordinates.

[0060] The spatiotemporal image portal is used to automatically generate a fixed area centered on the current location in response to user commands, guiding the user to capture image content in that area; when the user's device enters the fixed area later, the system generates a spatiotemporal window in the AR view through image registration and overlay technology to display historical images of that area;

[0061] Spatial vector graffiti is used to respond to the user's 3D drawing actions, generating vector graphics based on three-dimensional spatial coordinates, supporting free 3D creation, and storing and rendering graphic data in the form of three-dimensional vector coordinates;

[0062] The surface-anchored planar drawing is used to respond to the user's specification of the diagonal points of the physical surface and define a two-dimensional drawing plane. After the user completes 2D drawing in this plane, the system stores the brush strokes in the form of projected two-dimensional coordinates and adjusts the display synchronously with the change of the physical surface's viewing angle.

[0063] This application enables triggerable spatial information tags that support multimodal content including text, images, voice, and video. These tags are strongly correlated with spatial coordinates, exhibiting high information density and proactive interaction, ensuring a lasting correspondence between information and physical location. The spatiotemporal image portal automatically generates shooting areas and records historical images. Subsequent users can revisit these images through an AR spatiotemporal window, enhancing spatial memory. Image overlay is precise and easy to operate. Spatial vector graffiti supports free 3D creation, with graphics stored and rendered according to three-dimensional coordinates. Editing is flexible and distortion-free across multiple perspectives, breaking through the limitations of 2D drawing. Surface-anchored planar drawing defines the plane through diagonal points, and 2D content adjusts synchronously with the physical surface perspective, resulting in a strong sense of realism and adaptability to multiple scenarios such as walls and desktops. These four media cover all scenario needs, solving the problems of single interaction and weak immersion in traditional AR, and achieving a deep integration of virtual and real spatial digital expression.

[0064] User commands can be triggered either by application interface buttons or by predefined air gestures; spatial vector graffiti shares basic drawing properties with surface-anchored planar drawing, while spatial vector graffiti has its own independent advanced brush tool library.

[0065] This application provides an intuitive entry point for novice users or those using touchscreen devices. The click logic conforms to common user habits, ensuring that basic users can get started quickly. For AR glasses without touchscreen devices or when hands are busy, commands are triggered through natural gestures, allowing operation without touching the device and enhancing the immersive experience of virtual and real integration. The two methods complement each other, ensuring that the operation threshold is low enough while meeting the needs of advanced users for natural interaction, covering all scenarios from beginner to professional use.

[0066] The specific steps for building and expanding the spatial map within the positioning module are as follows:

[0067] Map initialization: For areas not covered by the map, the system prompts the user to become a spatial explorer; after the user clicks to start scanning, the client collects environmental data through the environmental awareness module. When the scan volume reaches a preset threshold, an initial spatial data packet is generated and uploaded to the cloud server; the cloud server creates new block records in the spatial map database.

[0068] User positioning and content loading: When a user enters the area later, the client device downloads the corresponding block data from the cloud server, captures the image in real time through the camera and matches it with the visual features of the block data; if the match is successful, AR positioning is performed and the digital markings of the area are loaded simultaneously.

[0069] Dynamic map expansion: If a user enters an uncovered sub-area of ​​an existing map area, the client device automatically triggers map expansion, collects new area data, generates a map expansion package, and uploads it to the cloud server; the cloud merges the map expansion package with the original block data and updates the spatial map;

[0070] Cloud-based map stitching: When a user's movement trajectory connects two independent spatial blocks, the client device uploads the connectivity trajectory data; the cloud server triggers a map fusion algorithm to calculate the spatial transformation relationship between the two blocks and stitch them together into a unified, coherent large spatial map.

[0071] This application prompts ordinary users to participate in crowdsourced mapping as space explorers. An initial data package is generated based on the condition that the scan volume meets the standard, which greatly reduces the cost of professional mapping and enables rapid coverage of blank areas. When subsequent users enter, accurate AR positioning is completed through visual feature matching, and digital markings of the area are loaded simultaneously to enhance the immersive experience of virtual and real interaction. For uncovered sub-areas of the mapped area, the system automatically triggers extended scanning and merges data to adapt to complex spaces and avoid coverage blind spots.

[0072] The map fusion algorithm is as follows:

[0073] The process involves identifying independent blocks to be merged, along with associated trajectory and point cloud data; preprocessing to remove point cloud noise and filter feature points; finding cross-block feature association points through coarse matching using a fast nearest neighbor search library and fine matching using random sampling consistency; optimizing the spatial transformation model based on the iterative nearest point algorithm to unify the block coordinate system; verifying and handling geometric and semantic conflicts in the splicing region; using a sparse BA algorithm to reduce accumulated errors for multiple blocks; generating a unified map and updating the database on the cloud server; and employing an anomaly handling mechanism to address matching failures and error exceeding limits.

[0074] This application preprocesses to remove point cloud noise and selects stable feature points, laying a high-quality data foundation for subsequent matching and avoiding matching deviations caused by noise interference; by combining FLANN coarse matching and RANSAC fine matching, the accuracy of cross-block feature association is greatly improved, false matching is reduced, and the reliability of block association anchor points is ensured.

[0075] The spatial transformation model optimized by the ICP algorithm can accurately unify the coordinate systems of different blocks, solving the problem of fragmented coordinate systems in crowdsourced data; geometric and semantic conflict verification ensures that the stitched map has no logical contradictions and fits the real environment; the sparse BA algorithm is used for global optimization of multiple blocks, which can reduce the cumulative error of local stitching and meet the needs of AR interaction; the anomaly handling mechanism effectively deals with matching failure and error exceeding the limit, improving the robustness of the algorithm.

[0076] The management module specifically includes:

[0077] Users can create public and private communities, and digital scratches are stored in categories by community ID. Client devices can switch communities via the top drop-down menu or a dedicated voice button, and the digital scratches of the current community are displayed after switching. Digital scratches are sorted by comprehensive score. The settings interface of the client device provides a slider and a number input box, allowing users to adjust the number of scratches displayed at one time, and provides a refresh button to update the content list.

[0078] This application supports the creation of public and private communities. Digital imprints are stored in categories according to community IDs, which not only meets the needs of open sharing but also ensures the security of private content, realizes the management of information in different circles, and avoids the mixing of content in different scenarios. The client can switch communities through the top drop-down menu or the dedicated voice button. The operation path is short and adaptable to multiple devices. Users can quickly switch perspectives without complicated steps, which is especially suitable for efficient interaction when both hands are busy.

[0079] The digital engraving processing module receives touch, gesture, voice, and button creation commands from handheld devices and AR devices via a multimodal interface. It matches the corresponding interactive medium to call sub-modules, processes high-frequency interactions between spatial information tags (text, images, audio, and video), spatiotemporal image portal area videos, vector graffiti and planar drawings, and positioning modules. Based on 6DoF pose data, it generates precise location tags: binding 3D coordinates and map feature points, generating circular areas with anchor points, and projecting planar coordinates. It integrates content and metadata into encrypted data packets, uploads them to the cloud via HTTPS, provides feedback to the user upon completion, and caches them locally.

[0080] The map index module connects to the map data package uploaded by the client, and removes invalid data through format and quality checks, and performs deduplication preprocessing; the index construction adopts a composite structure of spatial hierarchy and metadata association; a three-level spatial index is built based on R tree, binding the three-dimensional boundary of the block with feature points; the metadata index is associated with block attributes and supports multi-condition filtering.

[0081] High-concurrency data processing uses a hash consistency algorithm to distribute requests to 10-100 computing nodes; real-time decompression and format conversion are performed on map data packets uploaded by the client, and digital marking content is preprocessed by segmenting and storing. AI visual matching constructs a distributed feature index by synchronizing the core feature library of each block from the map index module; real-time environmental feature data uploaded by the client positioning module is received, and coarse and fine matching strategies are used to calculate feature overlap for matching; commercial billing includes personal billing, enterprise advertising billing, and billing risk control.

[0082] A crowdsourced digital information tracking method for augmented reality environments, comprising the following steps:

[0083] S1. After the client device is initialized, the environmental perception module collects environmental features and device pose data through sensors and transmits them to the positioning module.

[0084] S2. The positioning module builds an extended spatial map based on the environmental perception module and generates a data package to upload to the cloud map index module.

[0085] S3. When the user is creating content, the digital scratch processing module receives instructions to generate multimedia content, adds spatial tags by combining the data from the positioning module, and packages and uploads it to the cloud digital scratch module.

[0086] S4. The map index module verifies uploaded map data, creates indexes, and updates them dynamically.

[0087] S5, the digital marking module stores content and associates it with spatial information, and the management module filters markings by community ID;

[0088] S6, the operation and computing module performs map fusion, high-concurrency processing, AI visual matching and commercial billing, and finally the client completes the positioning, retrieves and renders the scratch content.

[0089] Example 1

[0090] Client devices: must have a camera, GPS, IMU, and support 6DoF tracking;

[0091] Cloud servers: must have spatial data indexing, high-concurrency processing, AI visual matching, and billing capabilities;

[0092] Startup process: When a user opens the APP, they enter the space matching mode. The first time they use the app, they request location, camera, microphone, and storage permissions.

[0093] Core logic: The system automatically matches the current location with an existing map. If the match is successful, the etched lines are loaded; if it fails, the user is prompted to become a space explorer.

[0094] Space Pioneer Mapping Process:

[0095] The user actively clicks to start scanning;

[0096] Provides real-time feedback during scanning, including the environment being scanned and the progress grid.

[0097] When the scanned volume reaches the preset threshold, a message will appear indicating that the initial spatial map has been successfully created. After that, the system will automatically expand the map as you move within the area.

[0098] The creation process of the four types of engravings:

[0099] Trigger a button click on a handheld device; VR / AR devices use gestures or controller menus.

[0100] Place definition information points, doodles, and drawings: A semi-transparent preview model appears, and the system prompts you to select a placement location;

[0101] The image circle automatically generates a preview centered on the user.

[0102] Completed via gestures, controller triggers, or voice commands;

[0103] The scratches are materialized and anchored, accompanied by visual highlights and audio cues.

[0104] like Figure 2 As shown in the figure, the digital scratch output effect after the user completes the 2D creation is clearly presented, showing the text "Li Xiaobai was here"; this content belongs to the surface-anchored planar drawing result, which has been bound to the physical space through the two-dimensional coordinates after projection, and can be adjusted synchronously with the change of the physical surface perspective. It is an intuitive manifestation of the deep binding between digital information and physical space.

[0105] like Figure 3 As shown, this image is the interface for 2D creation, with the core display of the input stage for surface anchoring plane drawing. The interface includes the text "Li Xiaobai was here" entered by the user and sets key interactive options: first, it prompts "Please confirm the canvas size", and then the user confirms "Drawing completed" through the "Yes / No" button, which fully restores the operation logic of completing 2D creation after the user defines the two-dimensional drawing plane.

[0106] like Figure 4 As shown, this image presents the 3D creation output effect of spatial vector graffiti, showcasing vector graphics generated based on three-dimensional spatial coordinates. The graphics are stored and rendered in the form of three-dimensional vector coordinates, representing the visualization result of the user's 3D painting actions. It supports multi-view viewing without distortion, reflecting the interactive characteristics of free 3D creation.

[0107] like Figure 5 As shown, this image is the creation confirmation interface for spatial vector graffiti. The core interaction logic is "Complete the creation of 3D marks" and provides two selection buttons: "Yes" and "No". The interface focuses on the final confirmation stage of 3D creation. Users complete the creation and submission of 3D marks through this interface, which is a key step in the 3D digital mark input process.

[0108] like Figure 6 As shown in the figure, the AR display effect of the spatiotemporal image portal is demonstrated. The spatiotemporal window is generated through image registration and overlay technology to present the historical images of the corresponding area. This window is the AR view content triggered when the user enters the preset fixed area, realizing the "spatiotemporal backtracking" function, allowing users to intuitively see the past images of the area and enhancing the immersive experience of AR social interaction.

[0109] like Figure 7As shown in the image, this is the creation operation guide interface of the spatiotemporal image portal, which includes the core prompt information: "A time window will be generated at the current location. Please face the correct perspective and keep it horizontal." The bottom displays the feedback prompt "Generation successful." The interface restores the complete process of the system guiding users to adjust the shooting perspective, complete the regional image acquisition, and generate the image successfully when creating the spatiotemporal image portal.

[0110] like Figure 8 As shown in the figure, the output effect of the spatial information tag can be triggered. The interface clearly presents the text content entered by the user, "The sunset here is really beautiful~", along with punctuation marks and layout format. This content is one of the multimedia contents entered by the user when creating the spatial information tag. Clicking the tag will display it, which reflects the interactive function of discrete information tagging.

[0111] like Figure 9 As shown, this image represents the text input interface for spatial information tags. It focuses on the input of user creative instructions. The interface includes a text input area, supporting users to enter text from multimedia content such as images, voice, and video. It is a concrete manifestation of the digital marking processing module receiving creative instructions through a multimodal interface, laying the foundation for the subsequent generation of spatial information tags with location labels.

[0112] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A crowdsourced digital information tracking system for augmented reality environments, characterized in that, The system includes: client devices and a cloud server, with data interaction between the client devices and the cloud server; the client devices include an environmental perception module, a positioning module, and a digital marking processing module; the cloud server includes a map indexing module, a digital marking module, a management module, and an operation computing module. The environmental perception module uses sensors to sense the environment and its own position, and collects environmental data. The positioning module processes environmental data locally in real time, builds and expands a spatial map of the environment, and then uploads it. The digital marking processing module receives user creation instructions, generates multimedia content based on interactive media, adds location tags to the content based on the spatial map output by the positioning module, and packages and uploads it to the digital marking module. The map indexing module is used to create and access indexes on the spatial map data uploaded by crowdsourcing. The digital etching module is used to store multimedia content created by users based on media, along with the spatial coordinates and duration of the content.

2. The crowdsourced digital information tracking system for augmented reality environments according to claim 1, characterized in that: The interactive media within the digital marking processing module include triggerable spatial information tags, spatiotemporal image portals, spatial vector graffiti, and surface-anchored planar drawings; Triggerable spatial information tags are created in response to user commands at specified spatial coordinates. Clicking them displays and allows input of multimedia content including text, audio, and video. The content is stored in the form of associated spatial coordinates. The spatiotemporal image portal is used to automatically generate a fixed area centered on the current location in response to user commands, guiding the user to capture image content in that area; when the user's device subsequently enters the fixed area, the system generates a spatiotemporal window in the AR view through image registration and overlay technology to display historical images of that area; Spatial vector graffiti is used to respond to the user's 3D drawing actions, generating vector graphics based on three-dimensional spatial coordinates, supporting free 3D creation, and storing and rendering graphic data in the form of three-dimensional vector coordinates; Surface-anchored planar drawing is used to respond to the user's specification of the diagonal points of the physical surface and define a two-dimensional drawing plane; after the user completes 2D drawing in this plane, the system stores the brush strokes in the form of projected two-dimensional coordinates and adjusts the display synchronously with the change of the physical surface's viewing angle.

3. The crowdsourced digital information tracking system for augmented reality environments according to claim 1, characterized in that: The specific steps for building and expanding the spatial map within the positioning module are as follows: Map initialization: For areas not covered by the map, the system prompts the user to become a spatial explorer; after the user clicks to start scanning, the client collects environmental data through the environmental awareness module. When the scan volume reaches a preset threshold, an initial spatial data packet is generated and uploaded to the cloud server; the cloud server creates new block records in the spatial map database. User positioning and content loading: When a user enters the area later, the client device downloads the corresponding block data from the cloud server, captures the image in real time through the camera and matches it with the visual features of the block data; if the match is successful, AR positioning is performed and the digital markings of the area are loaded simultaneously. Dynamic map expansion: If a user enters an uncovered sub-area of ​​an existing map area, the client device automatically triggers map expansion, collects new area data, generates a map expansion package, and uploads it to the cloud server; the cloud merges the map expansion package with the original block data and updates the spatial map; Cloud-based map stitching: When a user's movement trajectory connects two independent spatial blocks, the client device uploads the connectivity trajectory data; The cloud server triggers a map fusion algorithm to calculate the spatial transformation relationship between the two blocks and stitch them together into a unified and coherent large spatial map.

4. A crowdsourced digital information tracking system for augmented reality environments according to claim 1, characterized in that, The digital engraving processing module receives touch, gesture, voice, and button creation commands from handheld devices and AR devices via a multimodal interface. It matches the corresponding interactive medium to call sub-modules, processes high-frequency interactions between spatial information tags (text, images, audio, and video), spatiotemporal image portal area videos, vector graffiti and planar drawings, and positioning modules. Based on 6DoF pose data, it generates precise location tags: binding 3D coordinates and map feature points, generating circular areas with anchor points, and projecting planar coordinates. It integrates content and metadata into encrypted data packets, uploads them to the cloud via HTTPS, provides feedback to the user upon completion, and caches them locally.

5. A crowdsourced digital information tracking system for augmented reality environments according to claim 1, characterized in that, The map index module interfaces with the map data packets uploaded by the client, and removes invalid data through format and quality checks, and performs deduplication preprocessing; the index construction adopts a composite structure of spatial hierarchy and metadata association; a three-level spatial index is built based on R tree, binding the three-dimensional boundaries of blocks with feature points; Metadata indexes are associated with block attributes, supporting multi-condition filtering.

6. A crowdsourced digital information tracking system for augmented reality environments according to claim 1, characterized in that, The system also includes a management module, which specifically includes: Users can create public and private communities, and digital scratches are stored in categories by community ID. Client devices can switch communities via the top drop-down menu or a dedicated voice button, and the digital scratches of the current community are displayed after switching. Digital scratches are sorted by comprehensive score. The settings interface of the client device provides a slider and a number input box, allowing users to adjust the number of scratches displayed at one time, and provides a refresh button to update the content list.

7. A crowdsourced digital information tracking system for augmented reality environments according to claim 2, characterized in that: User commands can be triggered via application interface buttons or predefined air gestures; spatial vector graffiti shares basic drawing properties with surface-anchored planar drawing, while spatial vector graffiti has its own independent advanced brush tool library.

8. A crowdsourced digital information tracking system for augmented reality environments according to claim 3, characterized in that: The map fusion algorithm is as follows: Identify the independent blocks to be merged and their associated trajectory and point cloud data; Preprocessing removes point cloud noise and filters feature points; cross-block feature association points are found through coarse matching of a fast nearest neighbor search library and fine matching of random sampling consistency. The spatial transformation model is optimized based on the iterative nearest point algorithm, and the block coordinate system is unified. Verify and handle geometric and semantic conflicts in the spliced ​​areas; use the sparse BA algorithm to reduce accumulated errors for multiple blocks; generate a unified map and update the database on the cloud server; and use an exception handling mechanism to deal with matching failures and error exceeding limits.

9. A crowdsourced digital information tracking system for augmented reality environments according to claim 1, characterized in that: The system also includes an operations computing module to provide system computing support, including executing map fusion algorithms, high-concurrency data processing, AI visual matching, and business billing; High-concurrency data processing employs a hash consistency algorithm to distribute requests to 10-100 computing nodes; real-time decompression and format conversion are performed on map data packets uploaded by clients, and digital marking content is preprocessed by segmenting and storing data; AI visual matching constructs a distributed feature index by synchronizing the core feature library of each block from the map index module. It receives real-time environmental feature data uploaded by the client's positioning module, and uses coarse and fine matching strategies to calculate feature overlap for matching; commercial billing includes personal billing, enterprise advertising billing, and billing risk control.

10. A crowdsourced digital information tracking method for augmented reality environments, characterized in that, The specific steps are as follows: S1. After the client device is initialized, the environmental perception module collects environmental features and device pose data through sensors and transmits them to the positioning module. S2. The positioning module builds an extended spatial map based on the environmental perception module and generates a data package to upload to the cloud map index module. S3. When the user is creating content, the digital scratch processing module receives instructions to generate multimedia content, adds spatial tags by combining the data from the positioning module, and packages and uploads it to the cloud digital scratch module. S4. The map index module verifies uploaded map data, creates indexes, and updates them dynamically. S5, the digital marking module stores content and associates it with spatial information, and the management module filters markings by community ID; S6, the operation and computing module performs map fusion, high-concurrency processing, AI visual matching and commercial billing, and finally the client completes the positioning, retrieves and renders the scratch content.

Images