Spatial information display method and system based on intelligent wearable device

Abstract

The invention provides a spatial information display method based on an intelligent wearable device, and the method comprises the steps: S1, obtaining the perception data of a target object, and carrying out the object recognition and matching based on the perception data, so as to obtain the navigation information associated with the target object; s2, presenting introduction content to the user according to the sequence of the structured catalogue, and responding to an interaction request initiated by the user based on the target object in the navigation process; s3, in the execution process of the step S2, if an interrupt event causing the navigation process to be paused is detected, recording an interrupt context containing the current presentation state and the user identity information; s4, after the guide recovery trigger signal is detected, user identity information confirmation is carried out, and an interrupt context associated with the confirmed identity is retrieved; determining at least one content recovery node based on the interrupt context and the structured directory; and continuing to execute the step S2 according to the content recovery node.

Description

Technical Field

[0001] This application relates to the field of augmented reality technology, and in particular to a spatial information display method and system based on smart wearable devices. Background Technology

[0002] With the rapid evolution of Augmented Reality (AR) technology, its application in smart wearable devices (such as AR glasses and head-mounted displays) has penetrated into many important fields, including cultural tourism guidance, industrial inspection, education and training, and even medical assistance. By overlaying virtual information onto real-world scenes, AR systems provide users with real-time, contextualized interactive guidance and content enhancement, improving the intuitiveness and efficiency of information acquisition.

[0003] However, despite the promising prospects of combining AR technology with smart wearable devices, existing systems still face a series of technical bottlenecks in practical deployment and application, especially in terms of continuous experience, intelligent device perception, and environmental adaptability, which restricts their further popularization and the deepening of user experience.

[0004] Regarding task interruption and resumption mechanisms, existing AR guided tour systems rely primarily on smart wearable devices for interaction, such as voice commands, gesture recognition, or limited physical buttons. Compared to touch-enabled devices like smartphones, their operational channels are more limited, making it difficult to support complex status callbacks and progress selection. When the guided tour is interrupted by external interference, user-initiated interruptions, or temporary removal of the device, the system often cannot intelligently determine a resumption strategy based on the current scene, interruption duration, and user behavior history. It cannot automatically decide whether to start the guidance from the beginning or continue from the interruption point. Simultaneously, the system typically fails to trigger a timely content pause mechanism, leading to a disconnect between audio and visual cues and the user's actual needs, resulting in information omissions or temporal confusion. This not only disrupts the continuity of the immersive experience but also increases the user's operational burden, ultimately leading to decreased efficiency.

[0005] Furthermore, in terms of visual recognition and tracking, existing AR navigation systems lack stability under complex outdoor lighting conditions. In environments with backlighting, direct sunlight, low light, or drastic light fluctuations, the image quality captured by the camera deteriorates significantly, leading to a substantial decrease in recognition accuracy. This prevents the system from consistently and stably providing augmented information that is accurately aligned with the real-world scene, severely impacting the reliability of the navigation.

[0006] Therefore, there is an urgent need for a more intelligent, adaptive, and robust AR-based spatial information display method and system. Summary of the Invention

[0007] This application addresses the problems of insufficient intelligence and weak environmental adaptability of existing AR tour guide systems in real-world applications by providing a spatial information display method and system based on smart wearable devices. The aim is to improve the intelligent decision-making ability to resume tours after interruptions, the accurate perception of wearing status, and the robust recognition ability under complex lighting conditions, thereby achieving a smarter, more adaptive, and more reliable AR tour guide experience.

[0008] In a first aspect, embodiments of this application provide a spatial information display method based on a smart wearable device, including: Step S1: Obtain the perception data of the target object, and perform object recognition and matching based on the perception data to obtain the navigation information associated with the target object; the navigation information includes a structured directory organized according to a preset logic, and the introductory content corresponding to each node in the directory; Step S2: Present introductory content to the user according to the order of the structured catalog, and respond to the user's interactive requests based on the target object during the tour; Step S3: During the execution of step S2, if an interruption event that causes the tour flow to pause is detected, the interruption context containing the current presentation state and user identity information is recorded; the current presentation state includes the content of the directory nodes being presented and the interactive content. Step S4: After detecting the navigation recovery trigger signal, verify the user's identity information and retrieve the interruption context associated with the verified identity; based on the interruption context and the structured directory, determine at least one content recovery node; based on the content recovery node, continue to execute step S2.

[0009] In some embodiments, user identity information includes a fusion identifier generated based on at least two biometric features and a predefined social relationship group; User identity information verification includes: The current user's identity is verified based on the fused identifier; When the verification confirms that the current user is the previous user, the interrupt context associated with the previous user is directly retrieved. When the verification confirms that the current user is a new user, it is then determined whether the new user and the previous user belong to the same social relationship group; If they belong to the same social relationship group, the content recovery nodes provided for new users include those determined based on the interruption context of the previous user; If they do not belong to the same social relationship group, a content recovery node is provided for the new user, independent of the previous user's interruption context.

[0010] In some embodiments, the perceived data is image data; In step S1, the perceptual data of the target object is acquired, including: Acquire a preview image of the target object and determine the average brightness value of a predetermined central region in the preview image; the predetermined central region is the area in the center of the image that is 25% to 75% of the width and 25% to 75% of the height. Determine the corresponding exposure compensation parameters based on the average brightness value; Acquire the image of the target object to be identified based on the exposure compensation parameters; Image optimization processing is performed on the image to be recognized to obtain image data for object recognition and matching.

[0011] In some embodiments, determining the corresponding exposure compensation parameters based on the average brightness value includes: When the average brightness value is lower than the first underexposure threshold, the exposure compensation value is calculated according to the first underexposure compensation formula; When the average brightness value is higher than the first overexposure threshold, the exposure compensation value is calculated according to the first overexposure compensation formula. When the average brightness value is between the first underexposure threshold and the second underexposure threshold, the exposure compensation value is calculated according to the second underexposure compensation formula. When the average brightness value is between the second overexposure threshold and the first overexposure threshold, the exposure compensation value is calculated according to the second overexposure compensation formula. When the average brightness value is within the normal brightness range, maintain the current exposure settings.

[0012] In some embodiments, step S1, object recognition and matching based on perceptual data, includes: Extract feature vectors from image data; Calculate the similarity between the feature vector and the target object feature vector pre-stored in the vector database; If the similarity meets the predetermined conditions, the identification is deemed successful, and the navigation information associated with the identified target object is obtained.

[0013] In some embodiments, step S2 employs a state machine to manage the navigation process, and the state machine includes: The identification status is used to trigger and complete the identification of the target object; This section describes the playback status and is used to play the main introductory content corresponding to the target object. The interactive inquiry status is used to ask users whether they want to proceed to the extended content after the main introductory content has finished playing. Extended playback status, used to respond to the user's affirmative selection and play extended content; The associated introduction state is used to initiate an introduction to other objects associated with the target object; An open interactive state is used to respond to user-initiated interaction requests; The state machine switches between the above states according to preset transition rules.

[0014] In some embodiments, in step S3, the tour interruption event includes a voice interruption event and a wearing interruption event; Interruption handling for voice interruption events includes: During the presentation of the content, the voice capture service remains operational to monitor user voice input; When valid voice input is detected through the voice acquisition service, the presentation of the current introductory content is paused; Record the interruption context, which includes the current presentation state and user identity information; The interrupt handling for the wearing interrupt event includes: Real-time monitoring of the wearing status of smart wearable devices; When the device is detected to have been removed, pause the presentation of the current introductory content while keeping the voice capture service running; The record contains the interruption context, which includes the current presentation state and user identity information.

[0015] In some embodiments, presenting introductory content to the user in step S2 includes: Simultaneously receive streaming audio guides and corresponding text information; Text information is overlaid and displayed in the user's field of vision using augmented reality. The display sequence of text information is matched with the playback progress of the audio guide.

[0016] Secondly, embodiments of this application provide a spatial information display system based on a smart wearable device, employing a client-server architecture, including: Smart wearable device clients, which include: The image acquisition module is configured to acquire image data within the user's field of vision; The voice acquisition module is configured to acquire user voice input and transmit it to the server. The audio playback module is configured to play the navigation audio content; The display overlay module is configured to overlay navigation information onto the user's field of vision in an augmented reality manner; The process control module is configured to manage the status and interruption recovery logic of the guided tour process. The status management module is configured to manage the communication connection between the client and the server, as well as the device status. as well as The cloud server includes: An image recognition service is configured to receive image data, perform object recognition, and return the recognition results. The speech recognition service is configured to receive user voice input, perform speech recognition, and return the recognized text. The dialogue engine service is configured to generate dialogue responses based on the recognition results and the recognized text. The TTS synthesis service is configured to convert text responses into speech data and return them to the client; The vector database is configured to store the feature vectors and associated information of target objects for image recognition services to perform similarity matching.

[0017] Thirdly, embodiments of this application provide a smart wearable device, including the smart wearable device client of the aforementioned spatial information display system.

[0018] This application detects and records the interruption context, including progress, user identity, and interaction status, in real time. Upon resumption, it intelligently decides the recovery node based on identity verification and social relationships. When the same user resumes, seamless continuation is possible; when associated users on shared devices use the device, intelligent recommendations based on previous users' progress ensure a smooth and natural experience, achieving an intelligent and adaptive navigation interruption and recovery mechanism, significantly improving interaction continuity and user experience. It uses at least two biometric features to generate a fusion identifier, improving the accuracy and anti-counterfeiting capabilities of identity verification and effectively protecting user privacy. Combined with predefined social relationship groups, it enables personalized progress inheritance and content recommendations in device-sharing scenarios, enhancing system security and personalized service capabilities. The intelligent exposure compensation method analyzes the brightness of the central area of ​​the preview image and employs differentiated compensation strategies to automatically and quickly adapt to complex lighting conditions such as backlighting and low light, providing high-quality and stable input images for object recognition and improving the robustness and reliability of visual recognition in complex real-world environments. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a spatial information display method based on a smart wearable device, according to an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the spatial information display system based on a smart wearable device, according to an embodiment of this application. Detailed Implementation

[0021] The following specific embodiments illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Although the description of this application will be presented in conjunction with some embodiments, this does not mean that the features of this application are limited to this embodiment. On the contrary, the purpose of describing the application in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of this application. To provide a thorough understanding of this application, many specific details will be included in the following description. This application may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of this application, some specific details will be omitted in the description. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0022] It should be noted that in this specification, similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0024] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0025] In the description of this application, it should be understood that "electrical connection" can be understood as physical contact and electrical conduction between components; it can also be understood as the form of connection between different components in a circuit structure through physical lines that can transmit electrical signals, such as copper foil or wires on a printed circuit board (PCB). "Coupled through..." can be understood as electrical conduction through indirect coupling. Indirect coupling can be understood as contactless coupling. Those skilled in the art will understand that coupling refers to the phenomenon where there is a close cooperation and mutual influence between the inputs and outputs of two or more circuit elements or electrical networks, and energy is transferred from one side to the other through interaction. To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0026] In a first aspect, embodiments of this application provide a spatial information display method based on a smart wearable device. Figure 1 This is a schematic diagram of the overall process of the spatial information display method based on a smart wearable device provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps.

[0027] Step S1: Obtain the perception data of the target object, and perform object recognition and matching based on the perception data to obtain the navigation information associated with the target object. The navigation information includes a structured directory organized according to a preset logic, and the introductory content corresponding to each node in the directory.

[0028] Here, the target object is typically a real physical object that the user is currently interested in, such as museum exhibits, landmark buildings in scenic areas, merchandise displayed in shopping malls, plants in parks, or historical sites. Smart wearable devices continuously or on-demand collect perception data of the surrounding environment through their onboard cameras, microphones, or other sensors, with image data being the primary source of perception data.

[0029] In one implementation, the smart wearable device uses head-mounted AR glasses or lightweight smart glasses. When a user looks at an exhibit, the front-facing camera of the glasses captures a real-time image sequence containing the exhibit at a preset frame rate.

[0030] Specifically, step S1, which involves acquiring the perception data of the target object, further includes the following sub-steps.

[0031] Step S11: Obtain a preview image of the target object and determine the average brightness value of a predetermined central region in the preview image. The predetermined central region is the area with a width of 25% to 75% and a height of 25% to 75% of the center of the image.

[0032] Preview images are typically low-resolution, rapidly acquired reference frames used to determine if the current lighting conditions are suitable for capturing the image to be recognized. In practical applications, for example, when users view cultural relics in an indoor exhibition hall, the display case lighting may be strong or reflective; while in outdoor scenic areas, uneven lighting caused by direct sunlight or shade is common. Therefore, calculating the average brightness value of the central area first can more accurately reflect the lighting conditions of the subject the user is currently viewing, avoiding interference from overexposed or underexposed areas at the edges.

[0033] Step S12: Determine the corresponding exposure compensation parameters based on the average brightness value.

[0034] Exposure compensation parameters are used to guide subsequent image sensor adjustments to parameters such as exposure time, gain, or aperture to ensure that the image to be recognized achieves appropriate brightness distribution and contrast. In one embodiment, the device maintains a segmented exposure compensation map that divides the brightness range into multiple levels, each level corresponding to a different compensation strategy.

[0035] Furthermore, in step S12, if the average brightness value is lower than the first underexposure threshold, the exposure compensation value is calculated according to the first underexposure compensation formula; if the average brightness value is higher than the first overexposure threshold, the exposure compensation value is calculated according to the first overexposure compensation formula.

[0036] If the average brightness value is lower than the first underexposure threshold (for example, it can be set in the range of 60 to 80, which can be adjusted according to the characteristics of the image sensor), the current scene is determined to be severely underexposed. At this time, the system can calculate a large positive exposure compensation value to significantly improve the overall brightness of the image and restore details in dark areas. As a specific implementation, the exposure compensation value can be calculated according to the first underexposure compensation formula (110 - average brightness value) / 80. This strategy is suitable for shooting scenes of dark-colored cultural relics such as bronzes and jades in dimly lit museum exhibition halls, and can effectively improve image quality and provide clear input for subsequent recognition.

[0037] If the average brightness value is higher than the first overexposure threshold (typically 160-180), negative compensation is applied, mainly by shortening the exposure time to avoid highlight clipping. This is often encountered when photographing gilded Buddha statues or highly reflective modern sculptures. As a specific implementation method, the exposure compensation value can be calculated using the first overexposure compensation formula: (145 - average brightness value) / 80.

[0038] If the average brightness value is between the first underexposure threshold and the second underexposure threshold, the exposure compensation value is calculated according to the second underexposure compensation formula. If the average brightness value is between the second overexposure threshold and the first overexposure threshold, the exposure compensation value is calculated according to the second overexposure compensation formula. If the average brightness value is within the normal brightness range, the current exposure setting is maintained.

[0039] The second underexposure threshold is typically higher than the first underexposure threshold, forming a slightly underexposed range. Within this range, the compensation is gentler than the first underexposure compensation to avoid color distortion caused by over-brightening. Similarly, the slightly overexposed range also uses a smaller negative compensation magnitude. This tiered compensation strategy can achieve relatively balanced image quality under most everyday lighting conditions. As a specific implementation, the second underexposure compensation formula is (110 - average brightness value) / 100; the second overexposure compensation formula is (145 - average brightness value) / 100.

[0040] Step S13: Acquire the image of the target object to be identified according to the exposure compensation parameters, and perform image optimization processing on the image to be identified to obtain image data for object recognition and matching.

[0041] The acquired images to be identified are typically high-resolution single frames or short sequences of images. Image optimization processing includes, but is not limited to, conventional steps such as noise reduction, sharpening, color correction, and distortion correction. In one implementation, the image is cropped or scaled to a certain extent according to the expected size of the target object, so that the subject occupies an appropriate proportion of the image, thereby improving the efficiency and accuracy of subsequent feature extraction.

[0042] After image optimization is completed, the process moves on to object recognition and matching.

[0043] Step S14: Extract the feature vector of the image data.

[0044] Feature vector extraction utilizes pre-trained deep learning models (such as ResNet and EfficientNet) to extract high-dimensional feature vectors (e.g., 512-dimensional or 1024-dimensional) from images. These models have been optimized on datasets of annotated images of exhibits, artifacts, goods, and tourist attractions, and can output compact feature representations with strong discriminative power. In practical deployments, lightweight network structures or model quantization compression techniques are often employed to adapt to the computational limitations of smart wearable devices.

[0045] Step S15: Calculate the similarity between the feature vector and the target object feature vector stored in the vector database. If the similarity meets the predetermined conditions, the recognition is successful, and the navigation information associated with the recognized target object is obtained.

[0046] In one implementation, a vector database pre-stores a large number of standard feature vectors and their metadata for target objects. Within the vector database, the cosine distance between the feature vector of the image to be identified and the feature vectors of all objects in the database is calculated, and the result with the smallest cosine distance is returned. When the cosine distance is less than a preset threshold (e.g., 0.60–0.90), the identification is considered successful, and navigation information bound to the target object is retrieved from the database or a cloud-based content management system. When the cosine distance is greater than or equal to the preset threshold, the identification is deemed a failure, and the user is prompted to re-collect image data.

[0047] The navigation information includes a structured directory organized according to a pre-defined logic, and introductory content corresponding to each node in the directory. The structured directory typically uses a tree or directed acyclic graph format, with hierarchical organization being the most common, for example: First layer: Main body introduction (overview, historical background, basic features) Second layer: Detailed appreciation (materials and craftsmanship, artistic features, scientific value) The third layer: related knowledge (comparison of similar artifacts, historical context, cultural influence). Fourth layer: Interactive extensions (Frequently Asked Questions, Virtual Repair Demonstration, Voice Q&A, etc.) Each node corresponds to one or more introductory segments, which can be pre-recorded professional audio explanations or real-time synthesized speech-text pairs. The content length is usually controlled between 15 and 120 seconds, making it convenient for users to listen to in segments.

[0048] In one implementation, the navigation information package also includes metadata such as node navigation logic, recommendation weights, and priority tags, which are used to support flexible scheduling of the subsequent state machine.

[0049] Step S2: Present introductory content to the user according to the order of the structured catalog, and respond to the user's interactive requests based on the target object during the tour.

[0050] In practical use, the presentation of information to users usually adopts a combination of voice broadcasting as the main method and AR text overlay as a supplement, in order to reduce the excessive occupation of users' visual attention while ensuring the accessibility of information.

[0051] Specifically, step S2 further includes the following sub-steps.

[0052] Step S21: Simultaneously receive the streaming audio guide and corresponding text information.

[0053] The audio guide is typically streamed from local cache or delivered in segments from the cloud to avoid loading large files at once and causing startup delays. Text information arrives synchronously as timestamped subtitle segments, each segment corresponding to a sentence or a key narration from the audio.

[0054] Step S22: Text information is overlaid and displayed in the user's field of vision using augmented reality, wherein the display sequence of the text information matches the playback progress of the guide audio.

[0055] In AR glasses, text is typically displayed as semi-transparent white text on a black background or outlined text, floating in an appropriate position near the target object, such as directly above or to the right of an artifact, to avoid obscuring the subject itself. When the audio plays a certain sentence, the corresponding text segment gradually appears; when the audio is fast-forwarded or skipped, the text segment also fades out quickly or hides directly, thus achieving a high degree of synchronization between audio and text.

[0056] To manage the logical state of the entire tour guide process, a state machine is preferably used for scheduling.

[0057] The identification state is used to trigger and complete the identification of the target object; the introduction playback state is used to play the main introduction content corresponding to the target object; and the interactive inquiry state is used to ask the user whether to enter the extended content after the main introduction content has finished playing.

[0058] The recognition state is the entry point for the guided tour process. Once an object is successfully recognized, the introduction playback state begins. In the introduction playback state, the device plays audio node by node in the directory order, while simultaneously displaying the corresponding text in the field of view. After the main introduction (usually the first level node) has finished playing, the state machine switches to the interactive inquiry state, at which point it can announce via voice, "Would you like to learn more about the details of ××?" and wait for the user's response.

[0059] The extended playback state is used to respond to the user's affirmative selection and play extended content; the associated introduction state is used to initiate the introduction of other objects associated with the target object; and the open interaction state is used to respond to the user's interactive request. The state machine switches between the above states according to preset transition rules.

[0060] When a user responds affirmatively via voice or gesture, the device enters extended playback mode, continuing to play second, third, and other more in-depth content. If the user explicitly expresses a desire to view related objects (e.g., "Look at that tripod over there"), the device switches to related description mode, attempting to identify and load navigation information for nearby objects. In any non-recognition mode, users can initiate free interaction requests at any time, such as asking questions like "How was this made?", "How much is this worth?", or "How does it differ from the collections in the Palace Museum?" Upon receiving such requests, the device enters open interaction mode, invoking a question-and-answer engine or retrieval module to provide targeted answers.

[0061] Step S3: During the execution of step S2, if an interruption event that causes the tour process to pause is detected, the interruption context containing the current presentation state and user identity information is recorded. The current presentation state includes the content of the directory nodes being presented and the interactive content.

[0062] Interruption events are mainly divided into two categories: voice interruption events and wearable interruption events.

[0063] The handling process for voice interruption events is as follows.

[0064] Step S31: During the presentation of the introductory content, keep the voice acquisition service running to monitor the user's voice input. If valid voice input is detected through the voice acquisition service, pause the presentation of the current introductory content and record the interruption context containing the current presentation status and user identity information.

[0065] The voice acquisition service typically remains in a low-power, always-on state, using keyword wake-up or VAD (Voice Activity Detection) to determine if valid human speech is present. Once a user is detected speaking, such as "wait a minute," "stop a minute," or "don't speak yet," the current audio playback and text display are immediately paused, and the playback progress is recorded.

[0066] The interruption context includes at least the following information: the target object identifier of the current navigation, the path of the directory node that is playing or has just finished playing, the specific progress within the node (in seconds or sentence number), the content and answer of the most recent interaction request, and the current user's identity identifier (which may be a combination hash of the device's local account, bound mobile phone number, device unique ID and timestamp, etc.).

[0067] Step S32, the handling method for the wearing interruption event is as follows.

[0068] Smart wearable devices monitor wearing status in real time, typically using infrared proximity sensors, pressure sensors, capacitive sensors, or IMU motion data fusion to determine if the device is currently being worn. If the device is detected to have been removed, the ongoing audio and AR text presentation is immediately paused, while the voice capture service continues for a short period (usually 5-15 seconds) to prevent the user from briefly removing the device and then immediately putting it back on to continue.

[0069] If the device is not re-worn within a short period of time, the interruption context will be formally recorded. The content will be similar to that of a voice interruption, but the interruption type will be additionally marked as "wearing interruption".

[0070] Step S4: After detecting the navigation recovery trigger signal, the user's identity information is confirmed, and the interruption context associated with the confirmed identity is retrieved. Based on the interruption context and the structured directory, at least one content recovery node is determined. Based on the content recovery node, step S2 is continued.

[0071] The navigation system typically resumes operation in the following common scenarios: the user puts the device back on and remains stable for more than a preset time, such as three to eight seconds; the user actively wakes the device using a specific voice phrase, such as "continue speaking" or "let's continue"; the device detects that the user's gaze has been fixed on the same target object for an extended period of time; or the user triggers resumption through gestures, such as double-tapping a picture frame or making a specific wave. In actual products, these triggering methods are often monitored simultaneously to accommodate different usage habits and scenarios.

[0072] Once any one or more triggering conditions are met, the device first enters the user authentication phase. The core objective of this phase is to determine whether the current wearer is the same user as when the interruption occurred, or whether they belong to a group of related users who are allowed to share the navigation progress, thereby deciding whether to precisely continue the original progress or perform an appropriate context migration.

[0073] Step S41: Verify the identity of the current user based on the fusion identifier. If the verification determines that the current user is the previous user, the interrupt context associated with the previous user is directly retrieved. If the verification determines that the current user is a new user, it is determined whether the new user and the previous user belong to the same social relationship group.

[0074] In one implementation, when a user puts the device back on, the system first reads the most recent valid user fusion identifier and compares it with the currently collected real-time features. If the match exceeds a preset security threshold, it is directly determined to be the same user, and the most recent interruption context record is read directly from the local encrypted storage area or the cloud account space.

[0075] If the comparison result indicates a new user, the social relationship group assessment process will be further initiated. A social relationship group refers to a set of trust relationships pre-established by the user in the accompanying mobile application or cloud service, such as family member groups, couple groups, parent-child groups, friend-guided tour sharing groups, class study tour groups, company tour groups, etc. After obtaining authorization, each member within a group can choose whether to allow other members in the group to continue their guided tour progress.

[0076] Step S42: If the user belongs to the same social relationship group, the content recovery node provided to the new user includes the content recovery node determined based on the interruption context of the previous user. If the user does not belong to the same social relationship group, the content recovery node provided to the new user is independent of the interruption context of the previous user.

[0077] When a user is determined to belong to the same social relationship group, the system reads the previous user's interruption context and makes adaptive adjustments based on the current user's identity tags and device status. For example, in a family-oriented guided tour of a museum, if a parent interrupts the explanation of the "Painted General Figurine from Pit 2 of the Terracotta Warriors" at the third level of detail appreciation node, "Mineral Sources and Preservation Techniques of Painted Pigments," and the child puts on their glasses again and triggers a resumption, the system detects that both belong to the same parent-child group. It then prioritizes providing the resumption option starting from that node, while adding a gentle transitional phrase at the beginning of the voice broadcast, such as "Dad just talked about the mineral sources of the pigments, let's continue" or "Starting from the painting preservation techniques section," to reduce the child's sense of unfamiliarity when re-entering the tour.

[0078] In another typical scenario, such as a study tour, if the teacher interrupts the main introduction of the "Liangzhu Jade Cong King" at the end, and student A takes off the device to go to the restroom and then puts it back on, the system will recognize that student A belongs to the "Sixth Grade Class 3 Study Tour Group" created by the teacher. In this case, the system will allow the teacher to resume the tour from near the point where the teacher interrupted the tour, but can choose to backtrack slightly by 10 to 30 seconds to allow the student to re-establish the context. If student B belongs to a temporary group of another class, no continuation option will be provided. Instead, the tour will start a completely new tour from the point where the target object is re-identified.

[0079] If they do not belong to the same social relationship group, the system will reset the recovery node to the root node of the structured directory or the entry node of the most recently successfully identified target object by default, thereby ensuring the independence of personal privacy and the tour experience.

[0080] Step S43: Based on the content recovery node, continue with step S2.

[0081] Once a recovery node is determined, the system forcibly switches the current navigation state machine to the state corresponding to that node. For example, if the recovery node is the Nth sentence in the "Introduction Playback State," the audio segment with the start timestamp of that sentence and the corresponding AR subtitles are loaded directly, and playback begins from that position; if the recovery node is in the "Interactive Inquiry State," the inquiry statement is replayed and the user's response is awaited; if it is in the "Open Interaction State" and there are unfinished questions and answers, the system will prioritize attempting to continue the context of the previous question and answer session.

[0082] In actual operation, to prevent users from forgetting the content after a long interruption, the system can also insert a "quick review" audio clip of no more than 15 seconds before resuming playback. This audio clip condenses the key information from the previous sequence.

[0083] It should be noted that the content recovery node is not necessarily limited to the exact node at the time of interruption. In one implementation, the system provides multiple candidate recovery nodes for the user to choose from, based on the interruption duration, the user's historical behavior patterns, and the confidence level of the current target object.

[0084] In one implementation, when the interruption lasts less than 5 minutes, the system resumes directly from the point of interruption by default. When the interruption lasts between 5 and 30 minutes, two options are provided: "Resume from the point of interruption" and "Replay from the beginning of this section." When the interruption lasts more than 30 minutes, in addition to the above options, an additional "Replay from the beginning of the main introduction" option is provided as a conservative choice. This multi-option design can adapt to different users' memory retention abilities and attention recovery rates.

[0085] In one implementation, for a guide package with a deep directory hierarchy and long content, such as a guide to a single artifact in a large-scale special exhibition at the Palace Museum, which may contain six to eight layers of nodes, the system can dynamically determine the granularity of the selectable nodes based on the depth of the interruption location: shallow interruptions only provide options for the current node or the current layer, while deep interruptions can trace back to the nearest parent node or grandparent node, thereby avoiding the user having to start over at overly fragmented knowledge points, which would lead to a fragmented experience.

[0086] Continuing with the museum scenario as an example, when a user is viewing the "Fu Hao Owl-shaped Bronze Vessel," the interruption occurs at the fourth-level node, "Evolution of Vessel Shape and Antique Imitation Techniques." If the user triggers a recovery after 40 minutes, the system can provide the following three candidate nodes: Candidate node A: Continue directly from the 2-minute mark of the "Evolution of the Fu Hao Owl-shaped Zun and Antique Imitation Techniques" section; Candidate Node B: Restart from the beginning of the entire node "Evolution of Vessel Form and Antique Imitation Techniques"; Candidate Node C: Start from the beginning of the second layer of "Detail Appreciation" and quickly review the main body introduction and material craftsmanship in the first 30 seconds.

[0087] Users can make selections via voice commands such as "Choose the first one," "Go through the details from the beginning," or "Return to the main introduction," or through brief gestures. If a user does not make a selection within the preset waiting time, the system will default to the most conservative strategy, starting from the nearest parent node, thus balancing convenience and completeness.

[0088] In one implementation, in a social sharing scenario, when a new user connects, two additional strategies, "connection mode" and "parallel mode," can be provided.

[0089] In the continuation mode, new users fully inherit the playback progress, interaction history and state machine context of the previous user, which is suitable for situations where users with close relationships provide continuous explanations. In parallel mode, new users start from the same point but have independent playback progress and interaction records, which is suitable for semi-trust relationships or scenarios where each person needs to take their own study notes, such as between teachers and students, or between tour guides and tourists.

[0090] When parallel mode is implemented, the system maintains an independent context snapshot for each user in the cloud or locally, and marks the "shared source ID" and "derivative version number" in the tour information package metadata, thereby supporting subsequent progress fork management.

[0091] In one implementation, when the system detects that the same target object has been identified multiple times by multiple people within a short period of time and is associated with a social relationship group, it can proactively suggest "whether to continue the explanation from where ×× left off," and automatically resume the process after the user confirms, without requiring the user to manually trigger a resumption signal. This proactive recommendation mechanism significantly reduces repetitive operations and improves the overall continuity of the explanation.

[0092] After the recovery process in step S4 is completed, the state machine re-enters the presentation loop in step S2 according to the determined recovery node, continues to play the content in the structured directory order, and maintains the ability to respond to user interaction requests in real time.

[0093] This application's spatial information display method based on smart wearable devices detects and records interruption contexts containing progress, user identity, and interaction status in real time. Upon resumption, it intelligently decides the resumption node based on identity verification and social relationships. When the same user resumes, seamless continuation is possible; when associated users on shared devices use the device, intelligent recommendations based on previous users' progress ensure a smooth and natural experience, achieving an intelligent and adaptive navigation interruption and resumption mechanism, significantly improving interaction continuity and user experience. At least two biometric features are used to generate fused identifiers, improving the accuracy and anti-counterfeiting capabilities of identity verification and effectively protecting user privacy. Combined with predefined social relationship groups, personalized progress inheritance and content recommendations are possible in device-sharing scenarios, enhancing system security and personalized service capabilities. The intelligent exposure compensation method analyzes the brightness of the central area of ​​the preview image and employs differentiated compensation strategies, automatically and quickly adapting to complex lighting conditions such as backlighting and low light, providing high-quality and stable input images for object recognition, and improving the robustness of visual recognition and system reliability in complex real-world environments.

[0094] Secondly, embodiments of this application provide a spatial information display system based on a smart wearable device, employing a client-server architecture, including: a smart wearable device client, comprising: an image acquisition module configured to acquire image data in the user's field of vision; a voice acquisition module configured to acquire user voice input and transmit it to the server; an audio playback module configured to play guided audio content; a display overlay module configured to overlay guided information onto the user's field of vision in an augmented reality manner; a flow control module configured to manage the status and interruption recovery logic of the guided flow; and a status management module configured to manage the communication connection between the client and the server and the device status. The cloud-based service includes: an image recognition service configured to receive image data, perform object recognition, and return the recognition result; a speech recognition service configured to receive user voice input, perform speech recognition, and return the recognized text; a dialogue engine service configured to generate a dialogue response based on the recognition result and the recognized text; a TTS synthesis service configured to convert the text response into speech data and return it to the client; and a vector database configured to store the feature vectors and associated information of the target object for similarity matching by the image recognition service.

[0095] Thirdly, embodiments of this application provide a smart wearable device, including the smart wearable device client of the aforementioned spatial information display system.

[0096] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A spatial information display method based on a smart wearable device, characterized in that, include: Step S1: Obtain the perception data of the target object, and perform object recognition and matching based on the perception data to obtain the navigation information associated with the target object; The navigation information includes a structured directory organized according to a preset logic, and introductory content corresponding to each node in the directory; Step S2: Present the introductory content to the user according to the order of the structured directory, and respond to the user's interaction requests based on the target object during the navigation process; Step S3: During the execution of step S2, if an interruption event that causes the tour flow to pause is detected, the interruption context containing the current presentation state and user identity information is recorded; the current presentation state includes the directory node content and interactive content being presented. Step S4: After detecting the navigation recovery trigger signal, verify the user's identity information and retrieve the interruption context associated with the verified identity; Based on the interruption context and the structured directory, at least one content recovery node is determined; Restore the node based on the content and continue with step S2.

2. The method as described in claim 1, characterized in that, The user identity information includes a fusion identifier generated based on at least two biometric features and a predefined social relationship group; The user identity information confirmation includes: The current user's identity is verified based on the fused identifier; When the verification determines that the current user is the previous user, the interrupt context associated with the previous user is directly retrieved. When the verification confirms that the current user is a new user, it is then determined whether the new user and the previous user belong to the same social relationship group; If they belong to the same social relationship group, the content recovery node provided for the new user includes the content recovery node determined based on the interruption context of the previous user; If they do not belong to the same social relationship group, then the new user is provided with a content recovery node that is independent of the interruption context of the previous user.

3. The method as described in claim 1, characterized in that, The perceived data is image data; In step S1, the perceptual data of the target object is acquired, including: Acquire a preview image of the target object and determine the average brightness value of a predetermined central region in the preview image; the predetermined central region is an area that is 25% to 75% of the width and 25% to 75% of the height of the image center. Based on the average brightness value, determine the corresponding exposure compensation parameters; The image of the target object to be identified is acquired based on the exposure compensation parameters; The image to be identified is subjected to image optimization processing to obtain image data for object recognition and matching.

4. The method as described in claim 3, characterized in that, The step of determining the corresponding exposure compensation parameters based on the average brightness value includes: When the average brightness value is lower than the first underexposure threshold, the exposure compensation value is calculated according to the first underexposure compensation formula. When the average brightness value is higher than the first overexposure threshold, the exposure compensation value is calculated according to the first overexposure compensation formula. When the average brightness value is between the first underexposure threshold and the second underexposure threshold, the exposure compensation value is calculated according to the second underexposure compensation formula. When the average brightness value is between the second overexposure threshold and the first overexposure threshold, the exposure compensation value is calculated according to the second overexposure compensation formula. When the average brightness value is within the normal brightness range, maintain the current exposure settings.

5. The method as described in claim 3, characterized in that, In step S1, object recognition and matching are performed based on the perceived data, including: Extract the feature vector from the image data; Calculate the similarity between the feature vector and the target object feature vector pre-stored in the vector database; If the similarity meets the predetermined conditions, the identification is deemed successful, and the navigation information associated with the identified target object is obtained.

6. The method as described in claim 1, characterized in that, In step S2, a state machine is used to manage the tour process, and the state machine includes: The identification status is used to trigger and complete the identification of the target object; The playback status is displayed, which is used to play the main introductory content corresponding to the target object; The interactive inquiry state is used to ask the user whether to proceed to the extended content after the main introductory content has finished playing. Extended playback status, used to respond to the user's affirmative selection and play the extended content; The associated introduction state is used to initiate the introduction of other objects associated with the target object; An open interactive state is used to respond to user-initiated interaction requests; The state machine switches between the above states according to a preset transition rule.

7. The method as described in claim 1, characterized in that, In step S3, the tour guide interruption event includes a voice interruption event and a wearing interruption event; The interruption handling for the voice interruption event includes: During the presentation of the introductory content, the voice acquisition service remains operational to monitor user voice input; When valid voice input is detected through the voice acquisition service, the presentation of the currently presented content is paused; Record the interruption context, which includes the current presentation state and user identity information; The interruption handling for the wearing interruption event includes: Real-time monitoring of the wearing status of the smart wearable device; When the device is detected to have been removed, the presentation of the current introductory content is paused while the voice acquisition service continues to run; The record contains the interruption context, which includes the current presentation state and user identity information.

8. The method as described in claim 1, characterized in that, Step S2 presents introductory content to the user, including: Simultaneously receive streaming audio guides and corresponding text information; The text information is overlaid and displayed in the user's field of vision using augmented reality. The display sequence of the text information is matched with the playback progress of the guide audio.

9. A spatial information display system based on a smart wearable device, characterized in that, The architecture adopted is a client-server architecture, including: Smart wearable device clients, which include: The image acquisition module is configured to acquire image data within the user's field of vision; The voice acquisition module is configured to acquire user voice input and transmit it to the server. The audio playback module is configured to play the navigation audio content; The display overlay module is configured to overlay navigation information onto the user's field of vision in an augmented reality manner; The process control module is configured to manage the status and interruption recovery logic of the guided tour process. The status management module is configured to manage the communication connection between the client and the server, as well as the device status. as well as The cloud server includes: An image recognition service is configured to receive the image data, perform object recognition, and return the recognition result; The speech recognition service is configured to receive the user's voice input, perform speech recognition, and return the recognized text; The dialogue engine service is configured to generate a dialogue response based on the recognition result and the recognized text; The TTS synthesis service is configured to convert text responses into speech data and return them to the client. The vector database is configured to store the feature vectors and associated information of the target object for similarity matching by the image recognition service.

10. A smart wearable device, characterized in that, The smart wearable device client includes the spatial information display system of claim 9.

Images