An augmented reality and location-aware based scenic spot live-action tour interaction method and system

Abstract

The application discloses a kind of based on augmented reality and location-aware scenic spot real scene tour interaction method and system, it is related to augmented reality tour technical field;Real scene anchor point reference data is established to scenic spot tour interaction point;According to the positioning error range and the shooting direction of mobile terminal form candidate anchor point set, the consistency of the scale change of the cross-frame position of fixed real scene feature and terminal inertia motion is used to confirm real scene anchor point;Through candidate, confirmed, temporarily keep and lost state management virtual-real space conversion relationship, adjacent anchor point switching is completed, and near field communication label is used to assist anchor point recovery;According to the current virtual-real space conversion relationship, augmented reality content is superimposed and space interaction is executed.The application can reduce tour content error trigger, space drift and adjacent scenic spot repeated switching.

Description

Technical Field

[0001] This invention relates to the fields of augmented reality navigation, computer vision positioning, and mobile terminal location awareness. More specifically, this invention relates to a method and system for interactive real-scene navigation of scenic spots based on augmented reality and location awareness. Background Technology

[0002] With the development of mobile terminal camera, satellite positioning, posture perception and image processing capabilities, scenic spot virtual tour guides are gradually adopting augmented reality methods, overlaying attraction introductions, virtual characters, route prompts and interactive objects onto the real scene captured by the mobile terminal. Existing scenic spot tour guide systems usually pre-set geographical locations, content identifiers and trigger ranges for different attractions. After the mobile terminal enters the corresponding trigger range, the corresponding images, 3D models, audio or video content are retrieved according to the current location and orientation of the mobile terminal, and the retrieved content is mapped onto the real scene to form a tour guide interactive interface corresponding to the visitor's location.

[0003] However, attractions, street entrances, shops, and interactive facilities within scenic areas are usually densely distributed. Due to building obstruction, tree cover, terrain undulations, and multipath propagation of satellite signals, the location output by mobile terminals may fluctuate within a certain range, causing multiple adjacent guide points to simultaneously fall within the positioning error range of the current location. Some existing systems mainly determine the content to be displayed based on whether the current location enters a preset trigger range, making it difficult to determine the guide point corresponding to the actual shooting scene of the mobile terminal from multiple adjacent guide points. Other systems use single-frame real-scene images to identify attractions, but when adjacent buildings have similar appearances, real-scene objects partially repeat, tourists or vehicles obstruct the view, or ambient lighting changes, single-frame images are prone to forming unstable or incorrect matching results, resulting in augmented reality content that does not correspond to the actual attractions being retrieved and displayed.

[0004] Furthermore, as tourists continuously walk, turn, or change the shooting angle of their mobile devices within the scenic area, the visible features of the current real-world objects will constantly change and may temporarily disappear due to short-term occlusion. Existing systems typically directly re-determine the guide points and update the spatial location of the augmented reality content after changes in the location or image matching results. This can easily cause the displayed virtual objects to drift, jump, disappear, or repeatedly switch between adjacent guide points, and may also cause the same guide content to be triggered repeatedly. Therefore, existing technologies still lack a processing method that can continuously determine a unique and reliable real-world reference object and maintain a stable correspondence between the augmented reality content and the real scene when there are multiple guide points within the positioning error range, the real-world features are unstable for a short time, and tourists move continuously. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, such as the difficulty in continuously determining a unique and reliable real-scene anchor point when multiple adjacent guide points exist within the positioning error range, similar real-scene structures exist, real-scene features are temporarily obscured, and tourists move continuously, which easily leads to erroneous triggering of augmented reality content, spatial drift, or repeated switching between adjacent guide points, embodiments of the present invention provide a scenic area real-scene guide interaction method and system based on augmented reality and location awareness. This is achieved by pre-establishing a scenic area real-scene anchor system that includes location constraint areas, allowed viewing directions, fixed real-scene feature groups, content spatial correspondences, and spatial transformation relationships between adjacent anchor points. The system uses reference data to filter candidate real-scene anchor points based on the positioning error range and shooting direction of the mobile terminal. It confirms the real-scene anchor points by utilizing the consistency between the cross-frame changes of fixed real-scene features in continuous real-scene images and the terminal's inertial motion. Then, it controls the establishment, updating, maintenance, and restoration of the virtual-real space transformation relationship and the switching of adjacent real-scene anchor points based on the candidate state, confirmed state, temporarily held state, and lost state. This forms a closed-loop processing from real-scene anchor point establishment, candidate filtering, cross-frame verification, state management, adjacent switching, lost recovery to augmented reality display and interactive response, thereby solving the problems mentioned in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a scenic area real-scene guide interaction method based on augmented reality and location awareness, comprising the following steps: S1. Establish real-scene anchor points for multiple guide interaction points in the scenic area, and associate each real-scene anchor point with the corresponding position constraint area, allowed observation direction, fixed real-scene feature group, augmented reality content, content spatial correspondence between the augmented reality content and the fixed real-scene feature group, adjacent real-scene anchor points, anchor point switching area, and anchor point spatial transformation relationship between adjacent real-scene anchor points to form the scenic area real-scene anchor point benchmark data. S2. Obtain the current location of the mobile terminal and its positioning error range, terminal attitude data including terminal orientation and shooting direction, terminal inertial motion data and continuous real-scene image frames, compare the positioning error range with each position constraint region in the scenic area real-scene anchor point reference data, select real-scene anchor points where the position constraint region intersects with the positioning error range, and then filter according to the correspondence between the shooting direction and the allowed observation direction of each real-scene anchor point to form the current candidate anchor point set; S3. For each candidate real-scene anchor point in the current candidate anchor point set, match its fixed real-scene feature group with consecutive real-scene image frames to determine the actual position change and actual scale change of the matched fixed real-scene features in adjacent real-scene image frames. Based on the terminal inertial motion data, determine the expected change direction and expected change trend corresponding to the matched fixed real-scene features. Compare the actual position change and actual scale change with the expected change direction and expected change trend respectively. When the number of fixed real-scene features that meet the matching requirements is not less than the feature number threshold and the corresponding change difference does not exceed the motion consistency allowable amount, form an anchor point consistency verification result that the candidate real-scene anchor point meets the anchor point confirmation condition; otherwise, form an anchor point consistency verification result that the candidate real-scene anchor point does not meet the anchor point confirmation condition. S4. Manage the status of real-scene anchor points based on the anchor point consistency verification results formed within the continuous verification period. Mark candidate real-scene anchor points that have not yet met the anchor point confirmation conditions as candidate status, and mark candidate real-scene anchor points that continuously meet the anchor point confirmation conditions within the continuous verification period as confirmed status. Based on the matching fixed real-scene features and content space correspondence corresponding to the real-scene anchor points in the confirmed status, establish or update the current virtual-real space conversion relationship between augmented reality content and the current real-scene image frame. When a real-scene anchor point in the confirmed status has not yet met the anchor point confirmation conditions and has not reached the anchor point holding time limit, mark the real-scene anchor point as temporarily held status, and maintain the current virtual-real space conversion relationship based on the most recently confirmed virtual-real space conversion relationship and the motion increment corresponding to the terminal inertial motion data. When the duration of not meeting the anchor point confirmation conditions reaches the anchor point holding time limit, mark the real-scene anchor point as lost status. S5. When the mobile terminal enters the anchor point switching area associated with the current real-scene anchor point, the real-scene anchor point adjacent to the current real-scene anchor point is determined as the real-scene anchor point to be switched, and the anchor point consistency verification is performed on the real-scene anchor point to be switched according to S3. When the real-scene anchor point to be switched reaches the confirmed state, the current virtual-real space conversion relationship is converted to the anchor point coordinates of the real-scene anchor point to be switched according to the corresponding anchor point space conversion relationship, forming a switching reference space conversion relationship. The virtual-real space conversion relationship to be switched established according to the real-scene anchor point to be switched is compared with the switching reference space conversion relationship. When the space conversion difference between the two does not exceed the allowable amount of switching continuity, the real-scene anchor point to be switched is updated to the current real-scene anchor point and the switching of the current virtual-real space conversion relationship is completed. When the real-scene anchor point to be switched has not reached the confirmed state or the space conversion difference exceeds the allowable amount of switching continuity, the current real-scene anchor point and the current virtual-real space conversion relationship before the switch are maintained. S6. When the current real-scene anchor point is in a confirmed state or temporarily maintained state, the corresponding augmented reality content is superimposed onto the current real-scene image frame captured by the mobile terminal according to the current virtual-real space conversion relationship to form an augmented reality guide interactive screen; when the current real-scene anchor point is in a confirmed state and the received interactive input meets the spatial interaction conditions corresponding to the augmented reality content, the interactive response associated with the augmented reality content is executed, and the anchor point identifier, real-scene anchor point status and current virtual-real space conversion relationship of the current real-scene anchor point are used as the processing input for the next acquisition cycle, and the process returns to S2 to continue to perform candidate anchor point screening, anchor point consistency verification, real-scene anchor point status management and anchor point switching.

[0007] By adopting the above technical solution, the status of real-scene anchor points serves as the unified control basis for location perception, continuous image verification, virtual-real space conversion, augmented reality display, and interactive response. The current position and positioning error range of the mobile terminal are used to limit the real-scene anchor points that may correspond to the current real-scene image. The terminal's shooting direction is used to exclude real-scene anchor points that do not conform to the current observation direction. The consistency between the actual cross-frame changes of fixed real-scene features and the expected changes corresponding to the terminal's inertial motion is used to confirm the identity of the real-scene anchor point. The confirmed state is used to control the establishment and updating of the current virtual-real space conversion relationship. The temporarily maintained state is used to control the maintenance of spatial relationships during short-term recognition failures. The lost state is used to trigger re-screening or anchor point recovery, thereby avoiding the direct switching of augmented reality content based on a single location result or a single frame image result.

[0008] In a preferred embodiment, the formation of scenic area real-scene anchor point benchmark data in S1 includes the following steps: S11. Based on the geographical location of each guide interaction point, the area accessible to tourists, and the shooting range of the mobile terminal, establish a corresponding anchor point coordinate system, and determine the position constraint area and the allowed observation direction in the anchor point coordinate system to form anchor point position reference data. S12. Collect multiple sets of real-scene reference images of the same guide interaction point under different collection periods within the allowed observation direction. Extract candidate real-scene features from the real-scene reference images. Determine the candidate real-scene features that can be repeatedly matched under different collection periods and whose position changes do not exceed the allowable amount of feature stability as fixed real-scene features. Form a fixed real-scene feature group according to the positional relationship of each fixed real-scene feature in the anchor point coordinate system. S13. Determine the content reference position, content reference direction, and content display range of the augmented reality content in the anchor point coordinate system, and associate the content reference position, content reference direction, and content display range with the fixed real-scene feature group to form a content spatial correspondence. S14. Based on the location constraint area of ​​each real-scene anchor point and the area accessible to tourists, determine the adjacent real-scene anchor points, determine the overlapping area between adjacent real-scene anchor points that allows continuous observation or continuous passage as the anchor point switching area, and establish the anchor point spatial transformation relationship based on the position and direction relationship of adjacent real-scene anchor points under the unified geographical coordinates. S15. The anchor point location reference data, fixed real-scene feature groups, augmented reality content, content spatial correspondence, adjacent real-scene anchor points, anchor point switching areas, and anchor point spatial conversion relationships are associated and stored according to the real-scene anchor point identifiers to form the scenic area real-scene anchor point benchmark data.

[0009] By adopting the above technical solution, the position constraint area is used to represent the spatial range in which the mobile terminal may observe the corresponding navigation interaction point, and the allowed observation direction is used to represent the shooting direction range in which the mobile terminal can collect the corresponding fixed real-scene feature group. By comparing the repeated matching and position changes of candidate real-scene features under different collection periods, the long-term stable building outlines, door and window edges, signboard boundaries, sculpture structures or fixed facilities can be identified as fixed real-scene features, while excluding unstable objects such as pedestrians, vehicles, changing light and shadow and swaying vegetation. The content spatial correspondence is used to determine the display position, display direction and display range of augmented reality content relative to the fixed real-scene feature group, and the anchor point spatial transformation relationship is used to transform the spatial data under adjacent real-scene anchor points to the same anchor point coordinates for comparison.

[0010] In a preferred embodiment, forming the current candidate anchor point set in S2 includes the following steps: S21. Acquire the current location, positioning accuracy, terminal attitude data, terminal inertial motion data and continuous real-scene image frames of the mobile terminal according to their respective acquisition frequencies, and add acquisition timestamps to the acquired data respectively; S22. Based on the image acquisition timestamp of each consecutive real-scene image frame, associate the current positioning position, terminal attitude data and terminal inertial motion data within the preset time association range to form a time-series observation data group corresponding to each consecutive real-scene image frame, and mark the data that exceeds the preset time association range as invalid observation data. S23. Determine the positioning error range based on the current positioning location and the corresponding positioning accuracy, and select real-scene anchor points from the scenic real-scene anchor point reference data where the position constraint area intersects with the positioning error range to form a first candidate anchor point set. S24. Determine the shooting direction of the mobile terminal based on the terminal posture data, compare the shooting direction with the allowed observation direction of each real-scene anchor point in the first candidate anchor point set, remove the real-scene anchor points whose shooting direction does not fall into the corresponding allowed observation direction, and form a second candidate anchor point set. S25. The second candidate anchor point set is determined as the current candidate anchor point set, and the current candidate anchor point set is associated with the corresponding time series observation data group to form the current anchor point observation data.

[0011] By adopting the above technical solution, the image acquisition timestamp of the real-scene image frame is used as the correlation benchmark for position data, terminal attitude data and terminal inertial motion data, so that the data participating in the same verification cycle correspond to the movement and shooting state of the mobile terminal within the same time range; positioning accuracy is used to determine the positioning error range formed around the current positioning position. The first candidate anchor point set is formed by the intersection relationship between the position constraint area and the positioning error range, and the second candidate anchor point set is formed by the correspondence between the shooting direction and the allowed observation direction. Thus, real-scene anchor points whose position or observation direction does not conform to the current state are excluded before fixed real-scene feature matching.

[0012] In a preferred embodiment, the process of forming the anchor point consistency verification result in step S3 includes the following steps: S31. Based on the real-scene anchor point identifier of each candidate real-scene anchor point in the current candidate anchor point set, retrieve the corresponding fixed real-scene feature groups from the scenic area real-scene anchor point benchmark data respectively. S32. Match each fixed real-scene feature group with no less than three consecutive real-scene image frames to establish the correspondence between the same matched fixed real-scene feature in different real-scene image frames, and form the actual position change sequence and actual scale change sequence of each matched fixed real-scene feature according to the image acquisition timestamp. S33. Determine the translation direction, rotation direction, and translation trend of the mobile terminal based on the terminal inertial motion data between adjacent real-scene image frames, and form the expected change direction and expected change trend of the fixed real-scene features in adjacent real-scene image frames based on the translation direction, rotation direction, and translation trend. S34. Compare the actual position change sequence and actual scale change sequence of each matched fixed real scene feature with the corresponding expected change direction and expected change trend to determine the position change difference and scale change difference of each matched fixed real scene feature. S35. When the number of fixed real-scene features that meet the matching requirements is not less than the feature number threshold, and the differences in position and scale do not exceed the corresponding motion consistency allowable amount, an anchor point consistency verification result is formed indicating that the corresponding candidate real-scene anchor point meets the anchor point confirmation condition; otherwise, an anchor point consistency verification result is formed indicating that the corresponding candidate real-scene anchor point does not meet the anchor point confirmation condition.

[0013] By adopting the above technical solution, the same candidate real-scene anchor point needs to simultaneously meet the requirements of the number of fixed real-scene features and the requirements of cross-frame motion consistency. Among them, the number of fixed real-scene features is used to determine whether the current real-scene image contains enough features to characterize the candidate real-scene anchor point, and the differences in position and scale are used to determine whether the actual changes of fixed real-scene features in the real-scene image correspond to the translation, rotation or proximity state of the mobile terminal, thereby distinguishing adjacent guide interaction points with similar local structures but different actual positions.

[0014] In a preferred embodiment, managing the status of the real-world anchor point in S4 includes the following steps: S41. Mark the real-world anchor points that have entered the current candidate anchor point set but have not yet completed continuous verification as candidate states, and record the number of confirmation cycles for each candidate real-world anchor point to continuously meet the anchor point confirmation conditions. S42. When the number of confirmation cycles for any candidate real-scene anchor point that continuously meets the anchor point confirmation condition reaches the confirmation cycle threshold, the candidate real-scene anchor point is marked as confirmed and determined as the current real-scene anchor point. S43. Based on the position of the fixed real-scene feature corresponding to the current real-scene anchor point in the current real-scene image frame, the position of the fixed real-scene feature group in the anchor point coordinate system, and the content space correspondence, establish or update the current virtual-real space transformation relationship. The real-scene anchor point in the candidate state does not participate in the establishment or update of the current virtual-real space transformation relationship. S44. When the current real-scene anchor point fails to meet the anchor point confirmation condition within any verification cycle, it is changed from the confirmed state to the temporary holding state. The current virtual-real space conversion relationship is maintained according to the most recently confirmed virtual-real space conversion relationship and the terminal inertial motion increment during the temporary holding period. When the current real-scene anchor point meets the anchor point confirmation condition again within the anchor point holding time limit, it is restored to the confirmed state, and the current virtual-real space conversion relationship is updated according to the re-matched fixed real-scene features. S45. When the duration for which the current real-scene anchor point has not met the anchor point confirmation condition reaches the anchor point holding time limit, change its state from temporary holding to lost state, stop using the real-scene anchor point to update the current virtual-real space transformation relationship, and output the anchor point loss result.

[0015] By adopting the above technical solution, the candidate state is used to prevent real-scene anchors that have not yet completed continuous verification from participating in the current virtual-real space conversion relationship update. The confirmed state is used to indicate that the identity and spatial correspondence of the real-scene anchor have met the continuous confirmation requirements. The temporarily held state is used to maintain the most recent credible spatial relationship when tourists, vehicles or other objects temporarily occlude fixed real-scene features. The lost state is used to indicate that the current real-scene anchor can no longer be used as a credible spatial basis for augmented reality content. This avoids the change of matching results in a single verification cycle from directly causing virtual object drift, jump or incorrect switching.

[0016] In a preferred embodiment, the switching of the current real-scene anchor point is completed in S5, including the following steps: S51. Determine whether the mobile terminal has entered the anchor point switching area associated with the current real-scene anchor point based on the current location of the mobile terminal. When entering the anchor point switching area, determine the adjacent real-scene anchor point corresponding to the current real-scene anchor point based on the scenic area real-scene anchor point reference data, and determine the adjacent real-scene anchor point as the real-scene anchor point to be switched. S52. While maintaining the current real-scene anchor point and its current virtual-real space conversion relationship, perform anchor point consistency verification on continuous real-scene image frames according to the fixed real-scene feature group corresponding to the real-scene anchor point to be switched, and manage the real-scene anchor point status of the real-scene anchor point to be switched in accordance with S4. S53. When the real-scene anchor point to be switched reaches the confirmed state, establish the virtual-real space conversion relationship to be switched based on the matching fixed real-scene features and content space correspondence corresponding to the real-scene anchor point to be switched. S54. Based on the anchor point spatial transformation relationship between the current real-scene anchor point and the real-scene anchor point to be switched, the current virtual-real spatial transformation relationship is transformed to the anchor point coordinates of the real-scene anchor point to be switched, forming a switching reference spatial transformation relationship. The positional transformation difference and direction transformation difference between the virtual-real spatial transformation relationship to be switched and the switching reference spatial transformation relationship are determined respectively. S55. When the position conversion difference and direction conversion difference do not exceed the corresponding switching continuity allowable amount, update the real-scene anchor point to be switched to the current real-scene anchor point, and update the virtual-real space conversion relationship to be switched to the current virtual-real space conversion relationship; when any conversion difference exceeds the corresponding switching continuity allowable amount, continue to maintain the current real-scene anchor point and the current virtual-real space conversion relationship before the switch; when the current real-scene anchor point before the switch becomes lost and the real-scene anchor point to be switched has not yet reached the confirmed state, return to S2 to re-form the current candidate anchor point set.

[0017] By adopting the above technical solution, after the mobile terminal enters the anchor point switching area, the current real-scene anchor point before the switch is retained and the real-scene anchor point to be switched is verified to prevent immediate switching due to entering the location range of adjacent navigation interaction points; the current virtual-real space conversion relationship is converted to the anchor point coordinates of the real-scene anchor point to be switched through the anchor point spatial conversion relationship, so that the virtual-real space conversion relationship to be switched and the switching reference space conversion relationship can be compared under the same spatial benchmark. The switch is only completed when the identity of the real-scene anchor point to be switched has been confirmed and the position and direction conversion meet the continuity requirements.

[0018] In a preferred embodiment, S6 involves forming an augmented reality navigation interactive screen and executing an interactive response, including the following steps: S61. Based on the real-scene anchor point identifier of the current real-scene anchor point, retrieve the augmented reality content and content spatial correspondence associated with the current real-scene anchor point from the scenic area real-scene anchor point benchmark data. S62. When the current real-scene anchor point is in a confirmed state, the display position and display direction of the augmented reality content in the current real-scene image frame are determined by using the current virtual-real space transformation relationship; when the current real-scene anchor point is in a temporarily held state, the display position and display direction of the augmented reality content are determined by using the current virtual-real space transformation relationship maintained according to the terminal's inertial motion increment. S63. Superimpose augmented reality content onto the current real-world image frame according to the display position and display direction to form an augmented reality navigation interactive screen, and stop adding new interactive responses when the current real-world anchor point is lost. S64. Obtain the interactive input received by the mobile terminal, convert the screen position or terminal motion trajectory corresponding to the interactive input to the anchor coordinates of the current real-world anchor point, and determine whether the converted screen position or terminal motion trajectory enters the spatial interaction area corresponding to the augmented reality content. S65. When the current real-scene anchor point is in a confirmed state and the transformed screen position or terminal movement trajectory enters the spatial interaction area, execute the interactive response associated with the augmented reality content, update the interactive state of the augmented reality content, and pass the real-scene anchor point identifier, real-scene anchor point state, and current virtual-real space transformation relationship of the current real-scene anchor point to the next acquisition cycle.

[0019] By adopting the above technical solution, the augmented reality content display is maintained by utilizing the corresponding current virtual-real spatial transformation relationship in the confirmed state and the temporarily maintained state, respectively. The newly added interactive response is constrained by the real-scene anchor point state and the spatial interaction area. Even if the augmented reality content with established spatial relationship continues to be displayed in the temporarily maintained state, no new interactive response will be executed before the real-scene anchor point has been reconfirmed. The real-scene anchor point identifier, real-scene anchor point state, and current virtual-real spatial transformation relationship of the current real-scene anchor point continue to participate in the next acquisition cycle, thereby maintaining the continuous processing relationship between each acquisition cycle.

[0020] In a preferred embodiment, S5 further includes an anchor point recovery process based on near-field communication tags, the anchor point recovery process comprising the following steps: S56. When the current real-scene anchor point is lost and the mobile terminal reads the near-field communication tag set in the scenic area, obtain the tag identifier and tag reading time of the near-field communication tag. S57. Verify the label identifier according to the pre-stored correspondence between the label identifier, the label installation location and the real-scene anchor point identifier, and obtain the corresponding label installation location and target real-scene anchor point identifier after the label identifier passes the verification. S58. Based on the tag installation location, the positioning error range of the mobile terminal is redefined, and the target real-scene anchor point and the adjacent real-scene anchor points that intersect with the redefined positioning error range are determined as the set of recovery candidate anchor points. The target real-scene anchor point is marked as a candidate state instead of being directly marked as a confirmed state. S59. Obtain continuous real-scene image frames and terminal inertial motion data collected after the tag reading time, perform anchor point consistency verification on the target real-scene anchor point, when the target real-scene anchor point meets the anchor point confirmation condition, update it to the confirmed state and re-establish the current virtual-real space conversion relationship, when the target real-scene anchor point does not meet the anchor point confirmation condition, maintain its candidate state and do not trigger the corresponding augmented reality content interactive response.

[0021] By adopting the above technical solution, the near-field communication tag only provides the basis for location recovery and the target real-scene anchor point identification corresponding to the physical installation location, and does not directly replace the real-scene anchor point confirmation; after the terminal processor redefines the positioning error range according to the tag installation location, it still uses the continuous real-scene image frames after the tag reading time and the terminal inertial motion data to complete the cross-frame consistency verification of the target real-scene anchor point, thereby completing the physical location recovery and shooting posture confirmation respectively.

[0022] In the above embodiments, the feature stability allowance is used to limit the maximum positional change of the same candidate real-scene feature that can be determined as a fixed real-scene feature under different acquisition periods; the preset time correlation range is used to limit the maximum allowable acquisition time difference between the current positioning position, terminal attitude data, terminal inertial motion data and real-scene image frames; the feature quantity threshold is used to limit the minimum number of fixed real-scene features required for the candidate real-scene anchor point to meet the matching requirements; the motion consistency allowance is used to limit the maximum allowable difference between the actual positional change, actual scale change and corresponding expected change of the fixed real-scene feature; the confirmation period threshold is used to limit the number of consecutive times required for the candidate real-scene anchor point to become a confirmed state. The number of cycles for anchor point confirmation conditions; the anchor point holding time limit is used to limit the duration for which a confirmed real-world anchor point can continue to maintain the current virtual-real space transformation relationship after it has not yet met the anchor point confirmation conditions; the switching continuity allowance includes the position switching continuity allowance and the direction switching continuity allowance, which are used to limit the maximum position switching difference and the maximum direction switching difference allowed between the virtual-real space transformation relationship to be switched and the switching reference space transformation relationship, respectively; the above thresholds, allowances and time limits are predetermined based on the spatial scale of the scenic area's real-world anchor points, the distribution of fixed real-world features, the mobile terminal image acquisition frequency, the position perception accuracy, the inertial perception accuracy, the expected movement speed of tourists, and the display accuracy of augmented reality content.

[0023] By adopting the above technical solution, real-scene feature filtering, asynchronous data association, anchor point consistency verification, real-scene anchor point state transition and adjacent anchor point switching all have corresponding judgment boundaries, and each judgment result can be used as the input for subsequent steps, avoiding the direct determination of real-scene anchor points using only general conditions such as "successful recognition", "entering the trigger range" or "nearest location".

[0024] In another aspect, the present invention also provides a scenic spot real-scene guide interactive system based on augmented reality and location awareness, including a scenic spot data server and a mobile terminal that is communicatively connected to the scenic spot data server.

[0025] The scenic area data server includes an anchor point data storage device and a server processor. The anchor point data storage device stores real-scene anchor point reference data for the scenic area. The real-scene anchor point reference data includes position constraint areas associated with multiple real-scene anchor points, allowed viewing directions, fixed real-scene feature groups, augmented reality content, content spatial correspondence, adjacent real-scene anchor points, anchor point switching areas, and anchor point spatial transformation relationships. The server processor retrieves the corresponding real-scene anchor point reference data for the scenic area from the anchor point data storage device and sends it to the mobile terminal based on the current location or real-scene anchor point identifier sent by the mobile terminal.

[0026] The mobile terminal includes a location sensing component, an image acquisition component, an inertial sensing component, a display interaction component, and a terminal processor. The location sensing component acquires the current positioning position of the mobile terminal and its positioning error range. The image acquisition component acquires continuous real-scene image frames. The inertial sensing component acquires terminal attitude data and terminal inertial motion data. The terminal processor is connected to the location sensing component, the image acquisition component, the inertial sensing component, and the display interaction component.

[0027] The terminal processor compares the positioning error range with the position constraint regions of each real-scene anchor point, selects real-scene anchor points where the position constraint regions and positioning error ranges intersect, and then filters them according to the correspondence between the shooting direction and the allowed observation direction determined by the terminal attitude data to form a current candidate anchor point set. The terminal processor matches the fixed real-scene feature groups of each candidate real-scene anchor point in the current candidate anchor point set with consecutive real-scene image frames, and forms an anchor point consistency verification result based on the cross-frame actual position change and actual scale change of the matched fixed real-scene features and the expected change direction and expected change trend corresponding to the terminal inertial motion data. The terminal processor manages the real-scene anchor points as candidate state, confirmed state, temporarily maintained state, or lost state according to the anchor point consistency verification results within the continuous verification period. It only establishes or updates the current virtual-real space transformation relationship based on the real-scene anchor points in the confirmed state, and maintains the current virtual-real space transformation relationship in the temporarily maintained state based on the most recently confirmed virtual-real space transformation relationship and the terminal inertial motion increment.

[0028] After the mobile terminal enters the anchor point switching area, the terminal processor identifies the real-scene anchor point adjacent to the current real-scene anchor point as the real-scene anchor point to be switched and performs anchor point consistency verification on it. Based on the anchor point spatial transformation relationship between the current real-scene anchor point and the real-scene anchor point to be switched, the current virtual-real spatial transformation relationship is transformed to the anchor point coordinates of the real-scene anchor point to be switched, forming a switching reference spatial transformation relationship. The processor also determines the positional transformation difference and directional transformation difference between the virtual-real spatial transformation relationship to be switched established based on the real-scene anchor point to be switched and the switching reference spatial transformation relationship. When the real-scene anchor point to be switched reaches the confirmed state and the positional transformation difference and directional transformation difference do not exceed the corresponding switching continuity allowable amount, the terminal processor updates the real-scene anchor point to be switched to the current real-scene anchor point and completes the switching of the current virtual-real spatial transformation relationship. When the real-scene anchor point to be switched does not reach the confirmed state or any transformation difference exceeds the corresponding switching continuity allowable amount, the current real-scene anchor point and the current virtual-real spatial transformation relationship before the switch are maintained.

[0029] The display interaction component overlays the augmented reality content associated with the current real-world anchor point onto continuous real-world image frames based on the current virtual-real space conversion relationship output by the terminal processor, and transmits the received interactive input to the terminal processor. The terminal processor converts the screen position or terminal motion trajectory corresponding to the interactive input to the anchor point coordinates of the current real-world anchor point. When the current real-world anchor point is in a confirmed state and the converted screen position or terminal motion trajectory enters the spatial interaction area corresponding to the augmented reality content, it executes the interactive response and uses the real-world anchor point identifier, real-world anchor point status, and current virtual-real space conversion relationship for processing in the next acquisition cycle.

[0030] By adopting the above technical solution, the scenic area data server is responsible for providing the corresponding scenic area real-scene anchor point reference data according to the current location or real-scene anchor point identifier. The location sensing component, image acquisition component, and inertial sensing component of the mobile terminal provide the positioning error range, continuous real-scene image frames, terminal attitude data, and terminal inertial motion data, respectively. The terminal processor sequentially completes candidate anchor point screening, cross-frame consistency verification, management of four types of anchor point states, control of virtual-real space conversion relationship, and continuous switching of adjacent anchor points. The display interaction component executes augmented reality display and interaction according to the trusted spatial relationship formed by the terminal processor. Thus, the system structure, system acquisition data, and various online processing results correspond to the scenic area real-scene guide interaction method item by item.

[0031] In a preferred embodiment, the system further includes multiple near-field communication tags set within the scenic area, and the mobile terminal further includes a near-field communication reading component connected to the terminal processor; each near-field communication tag is set within the location constraint area or anchor point switching area of ​​the corresponding real-scene anchor point, and stores a tag identifier for distinguishing near-field communication tags; the anchor point data storage also stores the correspondence between the tag identifier, the tag installation location and the real-scene anchor point identifier.

[0032] When the current real-scene anchor point is lost and the near-field communication reading component reads the tag identifier, the terminal processor sends the tag identifier to the scenic area data server. The server processor verifies the tag identifier according to the correspondence and returns the corresponding tag installation location and target real-scene anchor point identifier to the mobile terminal. The terminal processor re-limits the positioning error range according to the tag installation location, marks the target real-scene anchor point as a candidate state, and performs anchor point consistency verification on the target real-scene anchor point based on the continuous real-scene image frames collected after tag reading and the terminal inertial motion data. Only after the target real-scene anchor point reaches the confirmed state is the current virtual-real space conversion relationship re-established and the interactive response of the corresponding augmented reality content restored.

[0033] By adopting the above technical solution, a data processing link is formed sequentially between the near-field communication tag, the near-field communication reading component, the scenic area data server, and the terminal processor. This link includes tag reading, tag verification, return of installation location and target anchor point, re-definition of positioning error range, target anchor point candidate marking, cross-frame consistency verification, reconstruction of current virtual-real space conversion relationship, and interactive response recovery. This makes the near-field communication recovery structure in the system correspond to the anchor point recovery process in the method.

[0034] The above technical solutions revolve around the establishment, screening, confirmation, state maintenance, adjacent switching, loss recovery, and augmented reality display and interaction of real-scene anchor points, forming the same inventive concept. The method solution fully implements the processing relationships of S1 to S6 and S11 to S65, and the system solution fully implements the scenic area data server, mobile terminal, and near-field communication recovery structure and corresponding processing procedures, thereby providing the corresponding inventive content basis for all the technical features in claims 1 to 10. This invention has at least the following effects and advantages: 1. By forming the current candidate anchor point set through the intersection relationship between the position constraint area and the positioning error range, and the correspondence between the shooting direction and the allowed observation direction, the real scene anchor points whose positions or observation directions do not conform to the current terminal state can be excluded before fixed real scene feature matching, thereby reducing the amount of data of irrelevant real scene anchor points participating in subsequent image processing.

[0035] 2. By confirming the consistency between the actual position and scale changes of fixed real-scene features in continuous real-scene image frames and the expected change direction and trend corresponding to the terminal's inertial motion, the real-scene anchor point is confirmed. This makes the confirmation of real-scene anchor points constrained by both the number of fixed real-scene features and the cross-frame motion relationship, which helps to reduce the probability of misconfirmation of anchor points when adjacent guide points are dense or the real-scene structure is similar.

[0036] 3. By controlling the establishment, updating, maintenance and stopping of the current virtual-real space transformation relationship through candidate state, confirmed state, temporarily held state and lost state, the failure of fixed real scene feature matching in a single verification cycle will not immediately cause the augmented reality content to disappear or switch, which helps to reduce the drift and jump of virtual objects caused by short-term occlusion, shooting blur or lighting changes.

[0037] 4. By converting the current virtual-real space conversion relationship to the anchor coordinates of the real-world anchor point to be switched through the anchor point spatial conversion relationship, the position and direction of the formed switching reference space conversion relationship are compared with the virtual-real space conversion relationship to be switched. This ensures that adjacent real-world anchor points are switched only when the anchor point identity has been confirmed and the spatial conversion is continuous, which helps to reduce repeated switching within the anchor point switching area.

[0038] 5. Near-field communication tags provide reliable tag installation locations and target real-world anchor point identifiers. Then, continuous real-world image frames read by the tags and terminal inertial motion data are used to confirm the target real-world anchor point. This establishes a sequential relationship between physical location recovery and real-world attitude confirmation, which helps improve the reliability of recovery after real-world anchor point loss and prevents tag reading from directly triggering erroneous augmented reality content.

[0039] 6. By distinguishing between the display conditions of augmented reality content and the conditions for adding interactive responses, the augmented reality content that has already established spatial relationships can continue to be maintained in the temporary holding state, while the new interactive response is only executed when the current real-world anchor point is in a confirmed state and the interactive input enters the corresponding spatial interaction area. This helps to prevent visitor operations from being mistakenly applied to other guide interaction points.

[0040] 7. By connecting and transmitting data between the scenic area data server, location sensing component, image acquisition component, inertial sensing component, display interaction component, and terminal processor, a continuous data processing link is formed, which includes real-scene anchor point reference data, mobile terminal observation data, anchor point consistency verification results, real-scene anchor point status, current virtual-real space conversion relationship, and interactive response results. This enables the system solution to implement the scenic area real-scene guide interaction method. Attached Figure Description

[0041] Figure 1 This is a flowchart of a scenic area real-scene guide interaction method based on augmented reality and location awareness in this invention.

[0042] Figure 2 This is a structural block diagram of a scenic area real-scene guide interactive system based on augmented reality and location awareness, as described in this invention. Detailed Implementation

[0043] Description of Implementation: The technical solutions in the embodiments of the present invention are clearly and completely described below. The described embodiments are used to illustrate the technical concept and specific implementation process of the present invention, rather than to limit the scope of protection of the present invention. Without departing from the technical concept of the present invention of maintaining a stable correspondence between augmented reality content and real scene with a reliable state of real scene anchor, equivalent substitutions made to data acquisition methods, threshold setting methods, hardware forms, feature expression methods and augmented reality content types can all be used to implement the present invention.

[0044] Terminology Explanation: The guide interaction point mentioned in this invention refers to a fixed location within a scenic area that has pre-set guide content or interactive content, including scenic buildings, street entrances, archways, sculptures, exhibition facilities, fixed shops, service facilities, and route turning points; the real-scene anchor point refers to a spatial benchmark established with reference to a fixed real-scene object near the guide interaction point that can be photographed by a mobile terminal and can be distinguished from other guide interaction points.

[0045] Description of fixed real-scene features: The fixed real-scene features described in this invention include building outlines, door and window edges, plaque boundaries, wall corners, sculptural structures, column edges, fixed facility outlines, or combinations of the above features; pedestrians, vehicles, movable exhibits, swaying vegetation, temporary advertisements, and light and shadow that change over time are not considered as the main features that determine the identity of the real-scene anchor point.

[0046] Spatial range description: The positioning error range described in this invention is a spatial range formed based on the current positioning position and positioning accuracy output by the mobile terminal. The spatial range can be represented as a circular area, elliptical area, polygonal area centered on the current positioning position, or an irregular area formed after intersecting with the area accessible to tourists. The allowed observation direction is one or more directional intervals that the mobile terminal can capture corresponding fixed real-scene feature groups.

[0047] Spatial relationship description: The content spatial correspondence described in this invention refers to the positional relationship between the augmented reality content's reference position, reference direction, and display range and the fixed real-world feature group in the anchor point coordinate system; the current virtual-real space transformation relationship is used to transform the augmented reality content in the anchor point coordinate system to the position and posture relationship under the current real-world image frame or mobile terminal display coordinates.

[0048] Judgment conditions description: The anchor point confirmation conditions described in this invention are conditions formed by jointly judging the number of fixed real-scene feature matches, the feature cross-frame position change, the feature cross-frame scale change, and the consistency of terminal inertial motion; the continuous verification period can be multiple consecutive real-scene image frames, multiple consecutive verification periods consisting of no less than three real-scene images, or multiple verification periods within a predetermined duration.

[0049] Threshold setting instructions: The feature quantity threshold, motion consistency allowance, confirmation cycle threshold, anchor point holding time limit, and switching continuity allowance described in this invention can be set according to the number of fixed real-scene features of the real-scene anchor point, the complexity of the scenic environment, the mobile terminal acquisition frequency, and the stability of the augmented reality display, and stored in the scenic real-scene anchor point reference data or the mobile terminal's operating parameters.

[0050] Example 1 Reference Figure 1 This embodiment provides a scenic spot real-scene guide interaction method based on augmented reality and location awareness. The method includes S1 establishing benchmark data of scenic spot real-scene anchor points, S2 forming a current candidate anchor point set, S3 forming an anchor point consistency verification result, S4 managing the status of real-scene anchor points and establishing or maintaining the current virtual-real space conversion relationship, S5 completing the switching of adjacent real-scene anchor points, and S6 forming an augmented reality guide interaction screen and executing the interaction response.

[0051] Specifically, S1 is executed first to establish a corresponding real-world anchor point for each guide interaction point in the scenic area. For each real-world anchor point, a location constraint area, allowed viewing direction, fixed real-world feature group, augmented reality content, content spatial correspondence, adjacent real-world anchor points, anchor point switching area, and anchor point spatial transformation relationship are associated. This forms the scenic area real-world anchor point benchmark data. The scenic area real-world anchor point benchmark data serves as the common data foundation for subsequent candidate anchor point screening, anchor point consistency verification, augmented reality content positioning, and adjacent anchor point switching.

[0052] After the mobile terminal enters the scenic area and starts the real-scene guide, S2 is executed to obtain the current positioning position, positioning error range, terminal attitude data, terminal inertial motion data and continuous real-scene image frames of the mobile terminal. First, based on the intersection relationship between the positioning error range and the position constraint area, real-scene anchor points that may be located around the mobile terminal are determined. Then, based on the correspondence between the shooting direction of the mobile terminal and the allowed observation direction of each real-scene anchor point, real-scene anchor points located behind the mobile terminal or not visible in the current shooting direction are excluded, thus forming the current candidate anchor point set.

[0053] After forming the current candidate anchor point set, S3 is executed. The fixed real-scene feature group of each candidate real-scene anchor point is matched with no less than three consecutive real-scene image frames to obtain the position and scale of each matched fixed real-scene feature in different real-scene image frames. Based on the terminal inertial motion data, the expected change direction and expected change trend of the fixed real-scene feature should occur when the mobile terminal is translated or rotated are determined. When the actual cross-frame change of the fixed real-scene feature matches the expected change, an anchor point consistency verification result is formed that the corresponding candidate real-scene anchor point meets the anchor point confirmation condition. Otherwise, an anchor point consistency verification result is formed that does not meet the anchor point confirmation condition.

[0054] After the anchor point consistency verification result is generated, S4 is executed to manage the candidate state, confirmed state, temporarily maintained state, and lost state of each real-scene anchor point. Only candidate real-scene anchor points that continuously meet the anchor point confirmation conditions can be converted to the confirmed state and used to establish or update the current virtual-real space transformation relationship. When a confirmed real-scene anchor point does not meet the anchor point confirmation conditions temporarily due to pedestrian occlusion, image blurring, or short-term lighting changes, the position of the augmented reality content is maintained using the most recently confirmed virtual-real space transformation relationship and the terminal inertial motion increment. When the duration of not meeting the anchor point confirmation conditions reaches the anchor point maintenance time limit, the corresponding real-scene anchor point is converted to the lost state.

[0055] When the mobile terminal enters the anchor point switching area, S5 is executed. While maintaining the current real-view anchor point and the current virtual-real space conversion relationship, the adjacent real-view anchor points to be switched are pre-verified. After the real-view anchor point to be switched reaches the confirmed state, the current virtual-real space conversion relationship is converted to the anchor point coordinates of the real-view anchor point to be switched according to the anchor point space conversion relationship, and compared with the virtual-real space conversion relationship to be switched re-established according to the real-view anchor point to be switched. Only when the position conversion difference and direction conversion difference of the two are within the allowable range, the switching of the real-view anchor point and the current virtual-real space conversion relationship is completed.

[0056] When the current real-world anchor point is in a confirmed state or a temporarily held state, S6 is executed. Based on the current virtual-real space transformation relationship, virtual characters, 3D models, attraction descriptions, route prompts, audio trigger markers, or other augmented reality content are overlaid onto the current real-world image frame. The temporarily held state is only used to maintain the already displayed augmented reality content and does not add new interactive responses that need to be based on the anchor point identity. The corresponding interactive response is only executed when the current real-world anchor point is in a confirmed state and the interactive input received by the mobile terminal meets the spatial interaction conditions corresponding to the augmented reality content.

[0057] Closed-loop processing description: S1 forms unified benchmark data for scenic spot anchor points; S2 narrows the identification range of real-scene anchor points; S3 verifies the consistency between changes in real-scene features and the movement of mobile terminals; S4 controls the confirmation, maintenance, and loss of real-scene anchor points; S5 controls the continuous switching between adjacent real-scene anchor points; and S6 completes the display of augmented reality content and spatial interaction. This forms a continuous guided tour closed loop in which candidate selection, anchor point verification, state management, spatial maintenance, anchor point switching, and interactive feedback are mutually reinforcing. After a data collection cycle is completed, the anchor point identifier, anchor point status, current virtual-real space conversion relationship, most recent anchor point consistency verification result, and corresponding data collection time of the current real-scene anchor point are passed to the next data collection cycle. Then, return to S2 to continue to perform candidate anchor point screening, anchor point consistency verification, real-scene anchor point status management, and adjacent anchor point switching, thus forming a continuously running scenic spot real-scene tour interactive closed loop.

[0058] Example 2: Establishment of Baseline Data for Scenic Area Anchor Points This embodiment describes the process of establishing the reference data of the real-scene anchor points in the scenic area (S1). The establishment process includes S11 establishing reference data for anchor point positions, S12 forming a fixed real-scene feature group, S13 forming a content spatial correspondence, S14 establishing the relationship between adjacent anchor points and anchor point spatial transformation, and S15 storing the reference data of the real-scene anchor points in the scenic area.

[0059] In S11, data producers establish an anchor point coordinate system based on the geographical location of each guide interaction point, the area accessible to tourists, and the shooting range of mobile terminals. The anchor point coordinate system can use a measured fixed point, the structural intersection of a fixed building, or other long-term stable real-world location as the origin of the coordinate system, and establish coordinate axes in the east, north, and vertical directions. It can also establish local coordinate axes based on the building facade direction or the main tourist traffic direction.

[0060] After the anchor point coordinate system is established, the location constraint area is determined based on the spatial range of the corresponding guide interaction point that the mobile terminal can stably capture, and the allowed observation direction is determined based on the visible direction of the fixed real-scene object relative to the area accessible to tourists. For example, for an archway that can only be viewed from the front square, the front square and its entrance passage are determined as the location constraint area, and the direction range facing the front of the archway is determined as the allowed observation direction, while the road behind the archway or the area blocked by other buildings are not included in the allowed observation range, thus forming anchor point location reference data.

[0061] In S12, multiple sets of real-scene reference images of the same guide interaction point are collected at different collection positions, different collection distances, and different collection periods within the allowed observation direction. The different collection periods can include morning, afternoon, sunny days, cloudy days, or different periods of visitor flow in the scenic area, so that the real-scene reference images contain reasonable changes in lighting and local occlusion.

[0062] Subsequently, corner points, edge intersections, contour turning points, line segment combinations, local texture descriptions, or fixed structure regions are extracted from each set of real-scene reference images as candidate real-scene features. The recurrence and relative positional stability of the same candidate real-scene feature in different real-scene reference images are compared. Candidate real-scene features that can be repeatedly matched in multiple acquisition periods and whose positional changes relative to the anchor point coordinate system or relative to other candidate real-scene features do not exceed the allowable amount of feature stability are identified as fixed real-scene features. Candidate real-scene features that only appear in a single period or change significantly with tourists, vehicles, light and shadow, and swaying vegetation are excluded.

[0063] After determining the fixed real-scene features, record the feature description data of each fixed real-scene feature, its position data in the anchor point coordinate system, its positional relationship relative to adjacent fixed real-scene features, the observable distance range, and the applicable observation direction, and form a fixed real-scene feature group according to the positional relationship. In one embodiment, a real-scene anchor point can set a fixed real-scene feature group covering the main observation direction. In another embodiment, a real-scene anchor point can set fixed real-scene feature subgroups for multiple allowed observation directions, and call the corresponding fixed real-scene feature subgroup according to the current shooting direction when the mobile terminal is running.

[0064] In S13, for augmented reality content that needs to be overlaid, the content reference position, content reference direction, and content display range are set in the anchor point coordinate system. For example, the virtual guide's feet are set at the ground reference position in front of the archway, the virtual guide's orientation is set to face the main entrance direction of tourists, and the height and horizontal range that the virtual guide is allowed to display are set, so as to avoid the virtual guide being displayed inside the building or floating above the ground.

[0065] Furthermore, the content reference position, content reference direction, and content display range are associated with the position of the fixed real-scene feature group in the anchor point coordinate system to form a content spatial correspondence. The augmented reality content can include 3D models, 2D animations, virtual characters, attraction names, text descriptions, route arrows, voice explanation trigger objects, video playback objects, and virtual interactive objects that can be operated by tourists. The same real-scene anchor point can be associated with one type of augmented reality content or multiple types of augmented reality content according to display conditions.

[0066] In S14, adjacent real-scene anchor points are determined based on the location constraint area of ​​each real-scene anchor point, the passable area for tourists, and the allowed observation direction. When a tourist can continuously pass from the location constraint area of ​​the first real-scene anchor point to the location constraint area of ​​the second real-scene anchor point, and can observe at least a portion of the fixed real-scene features of the first and second real-scene anchor points during the transition, the two are determined as adjacent real-scene anchor points, and the overlapping area where the two can continuously pass or continuously observe is determined as the anchor point switching area.

[0067] After determining adjacent real-scene anchor points, based on the position and orientation relationship between the two real-scene anchor points under unified geographic coordinates, the position transformation relationship and orientation transformation relationship between the anchor point coordinates of the first real-scene anchor point and the anchor point coordinates of the second real-scene anchor point are determined, thereby forming the anchor point spatial transformation relationship; the anchor point spatial transformation relationship can be established in advance through on-site measurement, or it can be corrected based on the fixed real-scene features that are commonly visible to the two anchor points and their positional relationship in the coordinate systems of the two anchor points.

[0068] In S15, using real-scene anchor point identifiers as indexes, the anchor point coordinate system, position constraint area, allowed observation direction, fixed real-scene feature group, augmented reality content identifier, content resource address, content spatial correspondence, adjacent real-scene anchor point identifiers, anchor point switching area, anchor point spatial transformation relationship, and various judgment thresholds are associated and stored to form the scenic area real-scene anchor point benchmark data. The scenic area real-scene anchor point benchmark data can be divided into multiple data packets according to the scenic area area, tour route, or data version, so that the scenic area data server can send data within the corresponding range according to the current location or current real-scene anchor point identifier of the mobile terminal.

[0069] Example 3: Formation of the current candidate anchor point set This embodiment describes the formation process of the current candidate anchor point set in S2. The formation process includes S21 collecting and labeling multi-source observation data, S22 forming a time-series observation data set, S23 forming a first candidate anchor point set, S24 forming a second candidate anchor point set, and S25 forming the current anchor point observation data.

[0070] In S21, the position sensing component acquires the current positioning position and positioning accuracy according to the first acquisition frequency, the image acquisition component acquires continuous real-scene image frames according to the second acquisition frequency, and the inertial sensing component acquires acceleration, angular velocity and terminal attitude data according to the third acquisition frequency, and adds an acquisition timestamp generated by the unified clock of the mobile terminal to each data. Since the acquisition frequencies of positioning, image and inertial data are usually different, the corresponding relationship is not directly established according to the data reception order, but the acquisition timestamp is used for time sequence association.

[0071] In one exemplary setup, the location data is collected at a frequency of one to five times per second, the real-scene image frames are collected at a frequency of fifteen to thirty frames per second, and the terminal inertial motion data is collected at a frequency of fifty to two hundred times per second. The above collection frequencies are only one example for implementing the present invention, and the actual collection frequencies can be determined according to the mobile terminal performance, the scenic area network conditions, and the augmented reality display refresh requirements.

[0072] In S22, based on the image acquisition timestamp of each real-scene image frame, data with timestamps within a preset time correlation range are selected from the location data, terminal attitude data, and terminal inertial motion data. For data whose acquisition time is earlier or later than the current image acquisition time but is still within the preset time correlation range, the observation value corresponding to the image acquisition time can be determined according to the changing trend of adjacent data. For data that exceeds the preset time correlation range, it is marked as invalid observation data and is not used for anchor point confirmation of the current image frame.

[0073] Therefore, a frame or a group of consecutive real-scene image frames, along with their corresponding current positioning location, positioning accuracy, terminal orientation, shooting direction, and terminal inertial motion data, are combined into a time-series observation data set. When a certain image frame lacks valid terminal inertial motion data, the image frame is not used to form a result that meets the anchor point confirmation conditions. Instead, the system waits for subsequent image frames with complete time-series observation data to continue verification, thereby avoiding the incorrect combination of data from different acquisition times.

[0074] In S23, a positioning error range is formed based on the current positioning location and the corresponding positioning accuracy. The positioning error range is then spatially intersected with each position constraint region in the scenic spot anchor point reference data. When the position constraint region of any real-scene anchor point intersects with the positioning error range, the real-scene anchor point is added to the first candidate anchor point set. When there is no intersection, the real-scene anchor point is excluded from the identification range of this collection cycle.

[0075] When the accessible area for tourists has clear boundaries, the positioning error range can be intersected with the accessible area for tourists corresponding to roads, squares, boardwalks or indoor passages. Then, the intersected range is compared with the constrained areas of each location, thereby reducing invalid candidate real-scene anchor points introduced by the satellite positioning results drifting into the interior of buildings, lakes or other inaccessible areas.

[0076] In S24, the shooting direction is determined based on the terminal orientation of the mobile terminal, the installation direction of the camera relative to the mobile terminal, and the current shooting posture. The shooting direction is then compared with the allowed observation direction of each real-scene anchor point in the first candidate anchor point set. When the shooting direction falls into the allowed observation direction of the corresponding real-scene anchor point, or when the deviation between the shooting direction and the allowed observation direction is within the preset allowed direction range, the real-scene anchor point is retained; otherwise, it is removed, thereby forming the second candidate anchor point set.

[0077] In S25, the second candidate anchor point set is determined as the current candidate anchor point set, and the real-scene anchor point identifier, fixed real-scene feature group, and corresponding time-series observation data group of each real-scene anchor point in the set are associated to form the current anchor point observation data. When the current candidate anchor point set is empty, augmented reality content is not triggered, and the position intersection judgment and observation direction judgment are re-executed in subsequent acquisition cycles. When there are multiple real-scene anchor points in the set, subsequent anchor point consistency checks are performed on multiple real-scene anchor points respectively, instead of directly determining the closest real-scene anchor point as the current real-scene anchor point.

[0078] Example 4: Anchor Point Consistency Verification This embodiment describes the process of determining the real-scene anchor point by S3 through the consistency between the changes of fixed real-scene features in continuous real-scene image frames and the actual movement of the mobile terminal. The process includes S31 retrieving a fixed real-scene feature group, S32 forming an actual position change sequence and an actual scale change sequence, S33 forming an expected change direction and an expected change trend, S34 determining the position change difference and the scale change difference, and S35 forming an anchor point consistency verification result.

[0079] In S31, based on the real-scene anchor point identifiers of each candidate real-scene anchor point in the current candidate anchor point set, the corresponding fixed real-scene feature groups are retrieved from the scenic area real-scene anchor point benchmark data respectively; when the same real-scene anchor point has multiple fixed real-scene feature subgroups corresponding to the observation direction, the corresponding subgroup is selected according to the current shooting direction, and fixed real-scene features that are not compatible with the current shooting distance or image clarity can be temporarily excluded.

[0080] In S32, the fixed real-scene feature group of each candidate real-scene anchor point is matched with no less than three consecutive real-scene image frames to determine the image position, feature scale and matching confidence of the fixed real-scene features in each real-scene image frame; then, based on the consistency of feature description, the continuity of image position and the structural relationship between adjacent features, a correspondence is established between the matching results from the same fixed real-scene structure in different real-scene image frames.

[0081] The correspondence is arranged according to the image acquisition timestamp to form the actual position change sequence and actual scale change sequence for each matching fixed real-scene feature. The actual position change sequence is used to represent the direction and degree of movement of the fixed real-scene feature in the horizontal and vertical directions of the image. The actual scale change sequence is used to represent the trend of magnification, reduction or remaining basically unchanged when the fixed real-scene feature moves closer to, further away from or changes the viewing angle of the mobile terminal.

[0082] In S33, the translation direction, rotation direction, and translation trend of the mobile terminal are determined based on the acceleration data, angular velocity data, and terminal attitude data collected between adjacent real-scene image frames. Based on this, the expected change direction and expected change trend of the fixed real-scene features in the image are determined. For example, when the mobile terminal rotates to the right, the fixed real-scene features in the image generally show an expected change direction of moving to the left. When the mobile terminal moves closer to the real-scene anchor point, the image spacing and feature scale between the fixed real-scene features generally show an increasing trend. When the mobile terminal remains stationary, the position and scale of the fixed real-scene features should remain stable within the allowable fluctuation range.

[0083] In S34, the actual position change sequence of each matching fixed real-scene feature is compared with the corresponding expected change direction, and the actual scale change sequence is compared with the corresponding expected change trend. For matching results with opposite change directions, significantly different change degrees, or only appearing in a single image frame, they are marked as not meeting the motion consistency requirements. For matching results with both change directions and change trends, they are marked as meeting the motion consistency requirements.

[0084] Furthermore, the differences in positional and scale variations of the matched fixed real-world features are statistically analyzed. The differences in positional variation can be determined based on the difference between the actual image movement direction and actual movement distance and the expected movement direction and expected movement range. The differences in scale variation can be determined based on the difference between the actual magnification or reduction trend and the expected trend. These differences are then compared with the corresponding motion consistency allowances.

[0085] In S35, when the number of fixed real-scene features that meet the matching requirements is not less than the feature number threshold, and the position change difference and scale change difference corresponding to the fixed real-scene features that meet the matching requirements do not exceed the corresponding motion consistency allowable amount, an anchor point consistency verification result is formed that the candidate real-scene anchor point meets the anchor point confirmation condition; when the number of fixed real-scene features is insufficient, or any type of change difference exceeds the corresponding motion consistency allowable amount, an anchor point consistency verification result is formed that the candidate real-scene anchor point does not meet the anchor point confirmation condition.

[0086] When multiple candidate real-scene anchors meet the anchor confirmation conditions within the same verification cycle, the verification priority order can be determined based on the number of fixed real-scene features that meet the matching requirements, the number of cycles that continuously meet the anchor confirmation conditions, the differences in position changes, the differences in scale changes, and the adjacency relationship with the current real-scene anchor in the previous collection cycle. Before any candidate real-scene anchor has completed continuous verification, it remains in the candidate state and does not directly trigger the corresponding augmented reality content based on the result of a single verification cycle.

[0087] Example 5: Real-world anchor point status management and virtual-real space conversion relationship update This embodiment S4 describes the management process of the real-scene anchor point status and the current virtual-real space conversion relationship. The process includes S41 recording the candidate anchor point confirmation cycle, S42 confirming the current real-scene anchor point, S43 establishing or updating the current virtual-real space conversion relationship, S44 maintaining and restoring the temporarily held state, and S45 outputting the anchor point loss result.

[0088] In S41, real-world anchors that have newly entered the current candidate anchor set and have not yet completed continuous verification are marked as candidate states, and their confirmation cycle count is updated after each verification cycle. When the candidate real-world anchor meets the anchor confirmation condition, the number of continuous confirmation cycles is increased. When it does not meet the anchor confirmation condition, the number of continuous confirmation cycles can be cleared to zero, or the number of confirmation cycles can be reduced according to the pre-set continuous verification rules.

[0089] In S42, when the number of confirmation cycles for any candidate real-scene anchor point that continuously meets the anchor point confirmation condition reaches the confirmation cycle threshold, it is changed from the candidate state to the confirmed state and determined as the current real-scene anchor point. In an exemplary embodiment, the confirmation cycle threshold is set to three consecutive verification cycles, each verification cycle containing at least three consecutive real-scene images, but the confirmation cycle threshold can be adjusted according to the image acquisition frequency and the complexity of the scenic area.

[0090] In S43, based on the position of the matched fixed real-scene feature corresponding to the current real-scene anchor point in the current real-scene image frame, the known position of the fixed real-scene feature group in the anchor point coordinate system, and the imaging parameters of the mobile terminal camera, the position and viewing direction of the mobile terminal camera relative to the real-scene anchor point are determined; then, based on the determined position and viewing direction and the content space correspondence of the augmented reality content, the current virtual-real space transformation relationship between the augmented reality content and the current real-scene image frame is established.

[0091] When the current real-scene anchor point continues to meet the anchor point confirmation conditions during subsequent verification cycles, the current virtual-real space conversion relationship is updated based on the newly matched fixed real-scene features to correct the cumulative error caused by the terminal's inertial motion integration. Real-scene anchor points in the candidate state only participate in identity verification and do not participate in the establishment or updating of the current virtual-real space conversion relationship, thereby avoiding the location jump of augmented reality content caused by unconfirmed real-scene anchor points.

[0092] In S44, when the current real-scene anchor point in the confirmed state fails to meet the anchor point confirmation condition within any verification cycle, the start time of the failure to meet the anchor point confirmation condition is recorded, and the current real-scene anchor point is converted to a temporary hold state. In the temporary hold state, the virtual-real space conversion relationship at the current acquisition time is calculated based on the most recently confirmed virtual-real space conversion relationship, according to the terminal translation increment and rotation increment after the starting point, and the calculated result is used to maintain the displayed augmented reality content.

[0093] When the current real-world anchor point meets the anchor point confirmation condition again within the anchor point holding time limit, it is restored from the temporarily held state to the confirmed state, and the current virtual-real space transformation relationship is updated using the newly matched fixed real-world features. During the update, the newly established spatial position can be continuously corrected with the spatial position maintained during the temporary holding period to avoid obvious jumps at the moment of restoration.

[0094] In S45, when the duration for which the current real-scene anchor point fails to meet the anchor point confirmation condition reaches the anchor point holding time limit, it is changed from a temporarily held state to a lost state, the use of the real-scene anchor point to update the current virtual-real spatial transformation relationship is stopped, and the anchor point loss result containing the real-scene anchor point identifier, the loss time, and the most recent valid spatial transformation relationship is output; in an exemplary embodiment, the anchor point holding time limit can be set to 0.5 seconds to 2 seconds, and the specific time limit is determined according to the image frame rate, the tourist walking speed, and the expected occlusion duration.

[0095] By transitioning between candidate state, confirmed state, temporarily held state, and lost state, a successful single-frame match cannot directly determine the current real-world anchor point, and a failed single-frame match will not directly cause the augmented reality content to disappear. Only when the continuous verification results meet the corresponding conditions will the anchor point identity be updated or the anchor point maintenance be terminated, thereby suppressing repeated switching between adjacent real-world anchor points.

[0096] Example 6: Continuous switching between adjacent real-world anchor points This embodiment describes the continuous switching process of adjacent real-scene anchor points in S5. The process includes S51 determining the real-scene anchor point to be switched, S52 verifying the real-scene anchor point to be switched in parallel, S53 establishing the virtual-real space conversion relationship to be switched, S54 forming the switching reference space conversion relationship and determining the conversion difference, and S55 completing the switching or maintaining the original real-scene anchor point.

[0097] In S51, spatial inclusion is determined based on the anchor point switching area associated with the current location of the mobile terminal and the current real-scene anchor point. When the current location falls into the anchor point switching area, or when it is determined that the mobile terminal is about to enter the anchor point switching area based on the direction of travel of the mobile terminal, the adjacent real-scene anchor points associated with the current real-scene anchor point are retrieved from the real-scene anchor point reference data of the scenic area, and the adjacent real-scene anchor points corresponding to the current shooting direction and the direction of travel of the tourists are determined as the real-scene anchor points to be switched.

[0098] In S52, the real-scene anchor point before the switch is kept as the current real-scene anchor point, and the augmented reality content is displayed using its current virtual-real space transformation relationship. At the same time, the fixed real-scene feature group corresponding to the real-scene anchor point to be switched is retrieved, the anchor point consistency check is performed on the continuous real-scene image frames, and the real-scene anchor point to be switched is managed according to the conversion rules of candidate state, confirmed state, temporarily held state and lost state.

[0099] In S53, when the real-scene anchor point to be switched continuously meets the anchor point confirmation condition and is converted to the confirmed state, the virtual-real space conversion relationship to be switched is established based on the position of the matching fixed real-scene feature of the real-scene anchor point in the current real-scene image frame, the position of the corresponding fixed real-scene feature in the coordinate system of the anchor point to be switched, and the content space correspondence.

[0100] In S54, based on the pre-established anchor point spatial transformation relationship between the current real-scene anchor point and the real-scene anchor point to be switched, the current virtual-real spatial transformation relationship before switching is transformed to the anchor point coordinates of the real-scene anchor point to be switched, thus obtaining the switching reference spatial transformation relationship; subsequently, the virtual-real spatial transformation relationship to be switched is compared with the switching reference spatial transformation relationship to determine the positional transformation difference and direction transformation difference between the two respectively.

[0101] In S55, when the difference in position conversion and the difference in direction conversion do not exceed the corresponding allowable amount of switching continuity, the real-scene anchor point to be switched is updated to the current real-scene anchor point, and the virtual-real space conversion relationship to be switched is updated to the current virtual-real space conversion relationship. During the switching process, the spatial conversion relationship before and after the switch can be continuously transitioned within multiple display frames so that the display position and direction of the augmented reality content maintain visual continuity.

[0102] When the real-scene anchor point to be switched has not yet reached the confirmed state, or when any difference between the virtual-real space conversion relationship to be switched and the switching reference space conversion relationship exceeds the corresponding allowable amount of switching continuity, the current real-scene anchor point and the current virtual-real space conversion relationship before the switch are maintained, and verification continues in subsequent verification cycles; when the current real-scene anchor point before the switch has been converted to the lost state and the real-scene anchor point to be switched has not yet reached the confirmed state, the current switching process is stopped and returned to S2, and the current candidate anchor point set is re-formed based on the latest positioning position, positioning error range and shooting direction.

[0103] Example 7: Anchor Point Recovery Based on Near Field Communication Tags This embodiment describes the near-field communication recovery process after the loss of the real-scene anchor point in S5. The recovery process includes S56 reading the near-field communication tag, S57 verifying the tag and obtaining the target real-scene anchor point, S58 forming a set of candidate anchor points for recovery, and S59 verifying and recovering the target real-scene anchor point.

[0104] During the establishment of the benchmark data for the scenic area's real-world anchor points, multiple near-field communication tags are set within the location constraint area or anchor point switching area of ​​the corresponding real-world anchor point. Each near-field communication tag stores a tag identifier used to distinguish the tag. The scenic area data server simultaneously stores the correspondence between the tag identifier, the tag installation location, and the target real-world anchor point identifier. The tag installation location can be represented using unified geographic coordinates and can also record its position in the target anchor point coordinate system.

[0105] In S56, when the current real-view anchor point is lost and the near-field communication reading component of the mobile terminal reads the near-field communication tag, the tag identifier and tag reading time are obtained, and the tag reading time is used as the starting time basis for subsequent recovery of image frames and terminal inertial motion data; when the current real-view anchor point is still in the confirmed state or temporarily maintained state, the tag reading event can be recorded, but the current real-view anchor point state is not forcibly changed.

[0106] In S57, the read tag identifier is verified according to the pre-stored correspondence between the tag identifier, the tag installation location, and the real-scene anchor point identifier. When the tag identifier does not exist, has expired, or is inconsistent with the current scenic area data version, the tag verification failure result is output, and the candidate anchor point set is not changed according to the tag. When the tag identifier passes the verification, the corresponding tag installation location and the target real-scene anchor point identifier are obtained.

[0107] In S58, the positioning error range of the mobile terminal is redefined using the tag installation location as a location reference. The redefined positioning error range is smaller than the original positioning error range formed by satellite positioning. Subsequently, the target real-scene anchor point and the adjacent real-scene anchor points that intersect with the redefined positioning error range are determined as the set of candidate anchor points for recovery. The target real-scene anchor point is marked as a candidate state, instead of being marked as a confirmed state directly because the tag identifier is read.

[0108] In S59, the continuous real-scene image frames and terminal inertial motion data collected after the tag reading time are acquired. The fixed real-scene feature group of the target real-scene anchor point is matched with the continuous real-scene image frames, and anchor point consistency verification and state management are performed according to S3 and S4. When the target real-scene anchor point continuously meets the anchor point confirmation condition, it is updated to the confirmed state, and the current virtual-real space conversion relationship is re-established according to the corresponding fixed real-scene features and content space correspondence.

[0109] When the target real-world anchor point does not meet the anchor point confirmation conditions, it remains in the candidate state or returns to S2 after the recovery verification time limit has expired, and does not trigger the corresponding augmented reality content interactive response; thus, the near-field communication tag is only used to provide a reliable location range and target anchor point identification, and the observation direction and spatial attitude of the real-world anchor point are still jointly confirmed by continuous real-world image frames and terminal inertial motion data.

[0110] Example 8: Augmented Reality Guided Tour and Spatial Interaction This embodiment describes the formation and interactive response process of the augmented reality guide screen in S6. The process includes S61 retrieving augmented reality content, S62 determining the content display position and display direction, S63 forming the augmented reality guide screen, S64 judging the spatial interaction conditions, and S65 executing the interactive response and transmitting the data for the next acquisition cycle.

[0111] In S61, based on the real-view anchor point identifier of the current real-view anchor point, the augmented reality content and content space correspondence associated with the current real-view anchor point are retrieved from the scenic area real-view anchor point reference data or the local cache of the mobile terminal; when the data volume of augmented reality content is large, the content resources can be pre-downloaded when the mobile terminal approaches the corresponding location constraint area, and the display is executed after the real-view anchor point reaches the confirmed state.

[0112] In S62, when the current real-scene anchor point is in a confirmed state, the content reference position and content reference direction of the augmented reality content are converted into the display position and display direction in the current real-scene image frame based on the current virtual-real space transformation relationship established or updated by matching fixed real-scene features; when the current real-scene anchor point is in a temporarily held state, the display position and display direction of the augmented reality content in the current real-scene image frame are determined based on the most recently confirmed virtual-real space transformation relationship and the terminal inertial motion increment during the temporary holding period.

[0113] In S63, augmented reality content is overlaid onto the current real-world image frame according to the determined display position, display direction, and content display range to form an augmented reality navigation interactive screen; when the current real-world anchor point is in a temporarily held state, the already triggered augmented reality content continues to be displayed, but the triggering of content that needs to re-determine the anchor point's identity can be paused; when the current real-world anchor point is lost, new interactive responses are stopped, and the already displayed augmented reality content can be faded out, frozen, or hidden.

[0114] In S64, interactive inputs formed by the mobile terminal through the touch screen, terminal posture changes, or movement trajectory are acquired; for touch input, the interaction direction from the mobile terminal to the real scene is determined according to the touch position and the imaging relationship of the mobile terminal camera, and the corresponding screen position is converted to the anchor coordinates of the current real scene anchor point using the current virtual-real space conversion relationship; for terminal motion input, the terminal motion trajectory is formed according to the continuous terminal posture and terminal inertial motion data, and the terminal motion trajectory is converted to the anchor coordinates of the current real scene anchor point.

[0115] Subsequently, it is determined whether the converted screen position, interaction direction, or terminal movement trajectory enters the spatial interaction area corresponding to the augmented reality content; the spatial interaction area can be set as a two-dimensional area around the augmented reality content, a three-dimensional spatial range, a predetermined proximity range, or an interaction range extending along a predetermined direction.

[0116] In S65, an interactive response associated with augmented reality content is executed only when the current real-world anchor point is in a confirmed state and the transformed screen position or terminal movement trajectory enters the corresponding spatial interaction area; the interactive response may include playing scenic spot explanations, controlling virtual character actions, switching scenic spot descriptions, displaying route information, starting Q&A content, capturing composite images, or updating the display status of interactive objects.

[0117] After completing the interactive response, update the interactive state of the augmented reality content, and pass the real-world anchor point identifier, real-world anchor point status, current virtual-real space conversion relationship, content interaction state and corresponding time to the next collection cycle; for interactive content that has been completed within the same real-world anchor point and is not allowed to be triggered repeatedly, it can prevent it from being triggered repeatedly during the short-term holding, reconfirmation or switching of adjacent anchor points based on the content interaction state.

[0118] Example 9: Interactive Real-Scene Guided Tour System for Scenic Spots Reference Figure 2 This embodiment provides a scenic area real-scene guide interactive system based on augmented reality and location awareness for implementing the above method. The system includes a scenic area data server and a mobile terminal that is communicatively connected to the scenic area data server. The mobile terminal can exchange data with the scenic area data server through a mobile communication network, a wireless local area network, or a scenic area-specific communication network.

[0119] The scenic area data server includes an anchor point data storage device and a server processor. The anchor point data storage device is used to store the scenic area's real-scene anchor point reference data according to the real-scene anchor point identifiers. The scenic area's real-scene anchor point reference data includes location constraint areas, allowed viewing directions, fixed real-scene feature groups, augmented reality content, content spatial correspondence, adjacent real-scene anchor points, anchor point switching areas, anchor point spatial conversion relationships, and corresponding judgment thresholds. The server processor retrieves the corresponding data from the anchor point data storage device and sends it to the mobile terminal based on the current location, scenic area identifier, or real-scene anchor point identifier sent by the mobile terminal.

[0120] The mobile terminal includes a location sensing component, an image acquisition component, an inertial sensing component, a display and interaction component, and a terminal processor. The location sensing component may include a satellite positioning receiver and a positioning accuracy output unit corresponding to the positioning result, used to obtain the current positioning position and its positioning error range. The image acquisition component includes a mobile terminal camera, used to continuously acquire real-scene image frames of the scenic area. The inertial sensing component may include an accelerometer, a gyroscope, and a magnetic field sensor, used to acquire terminal attitude data and terminal inertial motion data. The display and interaction component may include a touch screen or an augmented reality display, used to display augmented reality navigation interactive screens and receive interactive input.

[0121] The terminal processor is connected to the position sensing component, image acquisition component, inertial sensing component, and display interaction component, respectively, and obtains the real-scene anchor point reference data of the scenic area from the scenic area data server or local cache. The terminal processor forms the current candidate anchor point set based on the intersection relationship between the positioning error range and the position constraint area and the correspondence between the shooting direction and the allowed observation direction. Then, it forms the anchor point consistency verification result based on the cross-frame changes of fixed real-scene features and the terminal inertial motion data.

[0122] The terminal processor is also used to maintain the status records of each real-scene anchor point. Based on the anchor point consistency verification results within the continuous verification cycle, the real-scene anchor points are managed as candidate states, confirmed states, temporarily maintained states, or lost states. The terminal processor only establishes or updates the current virtual-real space conversion relationship based on the real-scene anchor points in the confirmed state, and maintains the current virtual-real space conversion relationship based on the most recently confirmed virtual-real space conversion relationship and the terminal inertial motion increment in the temporarily maintained state.

[0123] When the mobile terminal enters the anchor point switching area, the terminal processor maintains the current real-scene anchor point while performing anchor point consistency verification on adjacent real-scene anchor points, and forms a switching reference space conversion relationship based on the anchor point spatial conversion relationship. When the adjacent real-scene anchor points reach the confirmed state, and the position conversion difference and direction conversion difference between the virtual-real space conversion relationship corresponding to the adjacent real-scene anchor points and the switching reference space conversion relationship do not exceed the corresponding switching continuity allowable amount, the terminal processor completes the switching of the current real-scene anchor point and the current virtual-real space conversion relationship.

[0124] The display interaction component overlays the augmented reality content associated with the current real-world anchor point onto continuous real-world image frames based on the current virtual-real space transformation relationship output by the terminal processor, and transmits the received interactive input to the terminal processor. The terminal processor converts the interactive input to the anchor point coordinates of the current real-world anchor point, and executes the corresponding interactive response only when the current real-world anchor point is in a confirmed state and the interactive input meets the spatial interaction conditions.

[0125] In a further embodiment, the system also includes multiple near-field communication tags set within the scenic area, and the mobile terminal also includes a near-field communication reading component connected to the terminal processor; each near-field communication tag is set within the location constraint area or anchor point switching area of ​​the corresponding real-scene anchor point, and stores a tag identifier for distinguishing near-field communication tags, and the anchor point data storage also stores the correspondence between the tag identifier, the tag installation location and the real-scene anchor point identifier.

[0126] When the current real-scene anchor point is lost and the near-field communication reading component reads the tag identifier, the terminal processor sends the tag identifier and tag reading time to the scenic area data server. The server processor verifies the tag identifier and returns the corresponding tag installation location and target real-scene anchor point identifier to the mobile terminal. The terminal processor re-limits the positioning error range based on the tag installation location, marks the target real-scene anchor point as a candidate state, and uses the continuous real-scene image frames collected after tag reading and the terminal's inertial motion data to perform anchor point consistency verification. Only after the target real-scene anchor point reaches the confirmed state will the current virtual-real space conversion relationship be re-established and the interactive response of augmented reality content be restored.

[0127] Example 10: Complete Scenic Area Application Process This embodiment uses the first real-scene anchor point A and the second real-scene anchor point B, which are set up adjacently, as an example to illustrate the complete operation process of the present invention. The first real-scene anchor point A corresponds to the scenic area archway, and the second real-scene anchor point B corresponds to the fixed exhibition pavilion behind the archway. The visitor passage between the two real-scene anchor points is set as the anchor point switching area.

[0128] When establishing the benchmark data for the scenic area's real-scene anchor points, the structural intersection of the fixed pillar of the archway and the ground is used as the coordinate reference for the first real-scene anchor point A, and the fixed pillar at the center of the pavilion is used as the coordinate reference for the second real-scene anchor point B. Real-scene reference images are collected in the morning, afternoon, and at different times of visitor flow. The pillar edges, top outline, and plaque boundaries of the archway are determined as the first fixed real-scene feature group, and the eaves corners, pillar edges, and fixed door and window boundaries of the pavilion are determined as the second fixed real-scene feature group.

[0129] The virtual guide's content reference position is set on the ground in front of the archway, and the route arrow's content reference position is set on the passage leading to the pavilion. A content spatial correspondence relationship is established between the augmented reality content and the first fixed real-scene feature group. At the same time, an anchor point spatial transformation relationship is established based on the position and direction relationship of the first real-scene anchor point A and the second real-scene anchor point B under the same geographical coordinates, and the passage area that the two can observe continuously is set as the anchor point switching area.

[0130] After a tourist enters the front of the archway, the mobile terminal obtains the current location and the positioning error range. Under an exemplary operating condition, the positioning error range intersects with the position constraint areas of the first real-scene anchor point A and the second real-scene anchor point B. Therefore, both real-scene anchor points enter the first candidate anchor point set. Since the mobile terminal is currently facing the archway, the allowed observation direction of the first real-scene anchor point A is consistent with the shooting direction, while the second real-scene anchor point B is partially obscured by the archway. Therefore, after filtering by observation direction, the first real-scene anchor point A is retained as the current candidate real-scene anchor point.

[0131] The mobile terminal continuously captures real-scene image frames of the archway and matches the edges of the pillars, the top outline, and the boundary of the plaque with the first fixed real-scene feature group. When the tourist turns the mobile terminal to the right, the fixed real-scene features move to the left as a whole in the continuous images, and their actual movement direction corresponds to the terminal rotation direction detected by the gyroscope. When the number of fixed real-scene features that meet the matching requirements is not less than the feature number threshold, and the differences in position and scale do not exceed the corresponding motion consistency allowable amount, the first real-scene anchor point A meets the anchor point confirmation condition within the verification period.

[0132] In one exemplary parameter setting, the threshold for the number of fixed real-scene features that meet the matching requirements is set to eight, and the confirmation period threshold is set to three consecutive verification periods. When the first real-scene anchor point A meets the anchor point confirmation condition for three consecutive verification periods, it is changed from the candidate state to the confirmed state. The current virtual-real space conversion relationship is established according to the matching fixed real-scene features, anchor point coordinate system and content space correspondence, and the virtual guide is displayed at the predetermined ground position in front of the archway.

[0133] When a tourist briefly passes between the camera and the archway, obscuring some fixed real-world features, the first real-world anchor point A does not meet the anchor point confirmation condition within the current verification period. The mobile terminal converts it to a temporary hold state and continues to maintain the virtual guide's display position based on the most recently confirmed virtual-real space conversion relationship and the terminal's inertial motion increment during the obstruction period. When the obstruction ends and a fixed real-world feature is re-matched within the anchor point hold time limit, the first real-world anchor point A is restored to the confirmed state, and the current virtual-real space conversion relationship is corrected.

[0134] As visitors continue walking along the passage towards the pavilion and enter the anchor point switching area, the mobile terminal maintains the first real-scene anchor point A and its current virtual-real space conversion relationship, while continuously matching the eaves corner, column edge, and fixed door and window boundary of the second real-scene anchor point B. When the second real-scene anchor point B reaches the confirmed state, the virtual-real space conversion relationship to be switched is established based on the second real-scene anchor point B, and the switching reference space conversion relationship is formed based on the anchor point space conversion relationship between the first real-scene anchor point A and the second real-scene anchor point B.

[0135] In one exemplary setting, the allowable difference in position conversion is set to 0.8 meters, and the allowable difference in direction conversion is set to 8 degrees. When the difference in position conversion and the difference in direction conversion between the virtual-real space conversion relationship to be switched and the switching reference space conversion relationship do not exceed the corresponding allowable amounts, the second real-scene anchor point B is updated to the current real-scene anchor point, and the route arrow or the reference relationship of the virtual guide is continuously transferred to the second real-scene anchor point B.

[0136] If a tourist suddenly turns the camera towards the ground within the switching area, causing the first real-view anchor point A to be lost and the second real-view anchor point B not yet to be confirmed, then the second real-view anchor point B is not directly determined as the current real-view anchor point. Instead, the process returns to S2 to re-form the current candidate anchor point set. When the tourist uses a mobile terminal to read the near-field communication tag set next to the passage, the positioning error range is redefined according to the tag's installation location, and the second real-view anchor point B is marked as a candidate.

[0137] Subsequently, the mobile terminal acquires continuous real-scene image frames and terminal inertial motion data after the tag reading time. Only when the second fixed real-scene feature group matches the continuous real-scene image frames and its cross-frame changes are consistent with the terminal inertial motion, will the second real-scene anchor point B be converted to the confirmed state and the current virtual-real space conversion relationship be re-established. If the visitor does not point the camera towards the pavilion after reading the tag, the second real-scene anchor point B remains in the candidate state and no new interactive content dependent on the identity of the second real-scene anchor point B is displayed or triggered.

[0138] Once the second real-world anchor point B is confirmed and displays the virtual interactive object, visitors can click on the virtual interactive object on the display screen or move their mobile devices as prompted. The mobile device will then convert the click location or the device's movement trajectory to the anchor point coordinates of the second real-world anchor point B. When the conversion result enters the spatial interaction area corresponding to the virtual interactive object, the device will play explanations, perform virtual character actions, or update route prompts. The interaction status, along with the anchor point identifier, anchor point status, and current virtual-real spatial conversion relationship of the second real-world anchor point B, will be transmitted to the next data collection cycle.

[0139] As can be seen from the above implementation process, the present invention does not determine augmented reality content solely based on whether the mobile terminal enters the geographic trigger range. Instead, it first uses the positioning error range and shooting direction to form a set of candidate anchor points, and then uses the consistency between the changes in fixed real-scene features in continuous real-scene image frames and the terminal's inertial motion to confirm the unique and credible real-scene anchor point. The current virtual-real spatial conversion relationship is managed through candidate state, confirmed state, temporarily held state, and lost state. During the switching of adjacent real-scene anchor points and the recovery of near-field communication tags, the consistency verification of real-scene features and terminal motion is still used as the anchor point confirmation condition, thereby maintaining a stable spatial correspondence between augmented reality content and real scene under the conditions of positioning error, short-term occlusion, and continuous movement of tourists.

[0140] The acquisition frequency, feature quantity threshold, confirmation period threshold, anchor point holding time limit, position conversion difference allowance, and direction conversion difference allowance in the above embodiments are all exemplary parameters that can be adjusted according to the building density, guide interaction point spacing, mobile terminal performance, and real-scene feature stability of different scenic spots. As long as position constraints and observation direction are used to form a candidate range, the consistency between fixed real-scene feature cross-frame changes and terminal inertial motion is used to confirm the real-scene anchor point, and the display, holding, switching, and restoration of augmented reality content are controlled by the real-scene anchor point state and spatial continuity conditions, it belongs to the implementation of the technical solution of the present invention.

[0141] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method and system for interactive real-scene guided tours of scenic spots based on augmented reality and location awareness, characterized in that, Includes the following steps: S1. Establish real-scene anchor points for multiple guide interaction points in the scenic area, and associate each real-scene anchor point with the corresponding position constraint area, allowed observation direction, fixed real-scene feature group, augmented reality content, content spatial correspondence between the augmented reality content and the fixed real-scene feature group, adjacent real-scene anchor points, anchor point switching area, and anchor point spatial transformation relationship between adjacent real-scene anchor points to form the scenic area real-scene anchor point benchmark data. S2. Obtain the current location of the mobile terminal and its positioning error range, terminal attitude data including terminal orientation and shooting direction, terminal inertial motion data and continuous real-scene image frames, compare the positioning error range with each position constraint region in the scenic area real-scene anchor point reference data, select real-scene anchor points where the position constraint region intersects with the positioning error range, and then filter according to the correspondence between the shooting direction and the allowed observation direction of each real-scene anchor point to form the current candidate anchor point set; S3. For each candidate real-scene anchor point in the current candidate anchor point set, match its fixed real-scene feature group with consecutive real-scene image frames to determine the actual position change and actual scale change of the matched fixed real-scene features in adjacent real-scene image frames. Based on the terminal inertial motion data, determine the expected change direction and expected change trend corresponding to the matched fixed real-scene features. Compare the actual position change and actual scale change with the expected change direction and expected change trend respectively. When the number of fixed real-scene features that meet the matching requirements is not less than the feature number threshold and the corresponding change difference does not exceed the motion consistency allowable amount, form the anchor point consistency verification result of the candidate real-scene anchor point. S4. Manage the status of real-scene anchor points based on the anchor point consistency verification results formed within the continuous verification period. Mark candidate real-scene anchor points that have not yet met the anchor point confirmation conditions as candidate status, and mark candidate real-scene anchor points that continuously meet the anchor point confirmation conditions within the continuous verification period as confirmed status. Based on the matching fixed real-scene features and content space correspondence corresponding to the real-scene anchor points in the confirmed status, establish or update the current virtual-real space conversion relationship between augmented reality content and the current real-scene image frame. When a real-scene anchor point in the confirmed status has not yet met the anchor point confirmation conditions and has not reached the anchor point holding time limit, mark the real-scene anchor point as temporarily held status, and maintain the current virtual-real space conversion relationship based on the most recently confirmed virtual-real space conversion relationship and the motion increment corresponding to the terminal inertial motion data. When the duration of not meeting the anchor point confirmation conditions reaches the anchor point holding time limit, mark the real-scene anchor point as lost status. S5. When the mobile terminal enters the anchor point switching area associated with the current real-scene anchor point, the real-scene anchor point adjacent to the current real-scene anchor point is determined as the real-scene anchor point to be switched, and the anchor point consistency verification is performed on the real-scene anchor point to be switched according to S3; when the real-scene anchor point to be switched reaches the confirmed state, the current virtual-real space conversion relationship is converted to the anchor point coordinates of the real-scene anchor point to be switched according to the corresponding anchor point space conversion relationship, forming a switching reference space conversion relationship. The virtual-real space conversion relationship established based on the real-scene anchor point to be switched is compared with the switching reference space conversion relationship. When the difference between the two space conversions does not exceed the allowable amount of switching continuity, the real-scene anchor point to be switched is updated to the current real-scene anchor point and the switching of the current virtual-real space conversion relationship is completed. When the real-scene anchor point to be switched does not reach the confirmed state or the space conversion difference exceeds the allowable amount of switching continuity, the current real-scene anchor point and the current virtual-real space conversion relationship before the switch are maintained. S6. When the current real-world anchor point is in a confirmed state or temporarily maintained state, the corresponding augmented reality content is superimposed onto the current real-world image frame collected by the mobile terminal according to the current virtual-real space conversion relationship to form an augmented reality tour interactive screen. When the current real-world anchor point is in a confirmed state and the received interactive input satisfies the spatial interaction conditions corresponding to the augmented reality content, the interactive response associated with the augmented reality content is executed. The anchor point identifier, real-world anchor point status, and current virtual-real spatial conversion relationship of the current real-world anchor point are used as the processing input for the next acquisition cycle. Then, return to S2 to continue candidate anchor point screening, anchor point consistency verification, real-world anchor point status management, and anchor point switching.

2. The interactive method for scenic area real-scene navigation based on augmented reality and location awareness according to claim 1, characterized in that, In S1, the baseline data for the real-world anchor points of the scenic area is generated, including the following steps: S11. Based on the geographical location of each guide interaction point, the area accessible to tourists, and the shooting range of the mobile terminal, establish a corresponding anchor point coordinate system, and determine the position constraint area and the allowed observation direction in the anchor point coordinate system to form anchor point position reference data. S12. Collect multiple sets of real-scene reference images of the same guide interaction point under different collection periods within the allowed observation direction. Extract candidate real-scene features from the real-scene reference images. Determine the candidate real-scene features that can be repeatedly matched under different collection periods and whose position changes do not exceed the allowable amount of feature stability as fixed real-scene features. Form a fixed real-scene feature group according to the positional relationship of each fixed real-scene feature in the anchor point coordinate system. S13. Determine the content reference position, content reference direction, and content display range of the augmented reality content in the anchor point coordinate system, and associate the content reference position, content reference direction, and content display range with the fixed real-scene feature group to form a content spatial correspondence. S14. Based on the location constraint area of ​​each real-scene anchor point and the area accessible to tourists, determine the adjacent real-scene anchor points, determine the overlapping area between adjacent real-scene anchor points that allows continuous observation or continuous passage as the anchor point switching area, and establish the anchor point spatial transformation relationship based on the position and direction relationship of adjacent real-scene anchor points under the unified geographical coordinates. S15. The anchor point location reference data, fixed real-scene feature groups, augmented reality content, content spatial correspondence, adjacent real-scene anchor points, anchor point switching areas, and anchor point spatial conversion relationships are associated and stored according to the real-scene anchor point identifiers to form the scenic area real-scene anchor point benchmark data.

3. The interactive method for scenic area real-view navigation based on augmented reality and location awareness according to claim 2, characterized in that, In S2, the current candidate anchor point set is formed, including the following steps: S21. Acquire the current location, positioning accuracy, terminal attitude data, terminal inertial motion data and continuous real-scene image frames of the mobile terminal according to their respective acquisition frequencies, and add acquisition timestamps to the acquired data respectively; S22. Based on the image acquisition timestamp of each consecutive real-scene image frame, associate the current positioning position, terminal attitude data and terminal inertial motion data within the preset time association range to form a time-series observation data group corresponding to each consecutive real-scene image frame, and mark the data that exceeds the preset time association range as invalid observation data. S23. Determine the positioning error range based on the current positioning location and the corresponding positioning accuracy, and select real-scene anchor points from the scenic real-scene anchor point reference data where the position constraint area intersects with the positioning error range to form a first candidate anchor point set. S24. Determine the shooting direction of the mobile terminal based on the terminal posture data, compare the shooting direction with the allowed observation direction of each real-scene anchor point in the first candidate anchor point set, remove the real-scene anchor points whose shooting direction does not fall into the corresponding allowed observation direction, and form a second candidate anchor point set. S25. The second candidate anchor point set is determined as the current candidate anchor point set, and the current candidate anchor point set is associated with the corresponding time series observation data group to form the current anchor point observation data.

4. The scenic area real-scene guided interactive method based on augmented reality and location awareness according to claim 3, characterized in that, In S3, the anchor point consistency verification result is generated, including the following steps: S31. Based on the real-scene anchor point identifier of each candidate real-scene anchor point in the current candidate anchor point set, retrieve the corresponding fixed real-scene feature groups from the scenic area real-scene anchor point benchmark data respectively. S32. Match each fixed real-scene feature group with no less than three consecutive real-scene image frames to establish the correspondence between the same matched fixed real-scene feature in different real-scene image frames, and form the actual position change sequence and actual scale change sequence of each matched fixed real-scene feature according to the image acquisition timestamp. S33. Determine the translation direction, rotation direction, and translation trend of the mobile terminal based on the terminal inertial motion data between adjacent real-scene image frames, and form the expected change direction and expected change trend of the fixed real-scene features in adjacent real-scene image frames based on the translation direction, rotation direction, and translation trend. S34. Compare the actual position change sequence and actual scale change sequence of each matched fixed real scene feature with the corresponding expected change direction and expected change trend to determine the position change difference and scale change difference of each matched fixed real scene feature. S35. When the number of fixed real-scene features that meet the matching requirements is not less than the feature number threshold, and the differences in position and scale do not exceed the corresponding motion consistency allowable amount, an anchor point consistency verification result is formed indicating that the corresponding candidate real-scene anchor point meets the anchor point confirmation condition; otherwise, an anchor point consistency verification result is formed indicating that the corresponding candidate real-scene anchor point does not meet the anchor point confirmation condition.

5. The interactive method for scenic area real-scene navigation based on augmented reality and location awareness according to claim 4, characterized in that, In S4, managing the status of the real-world anchor points includes the following steps: S41. Mark the real-world anchor points that have entered the current candidate anchor point set but have not yet completed continuous verification as candidate states, and record the number of confirmation cycles for each candidate real-world anchor point to continuously meet the anchor point confirmation conditions. S42. When the number of confirmation cycles for any candidate real-scene anchor point that continuously meets the anchor point confirmation condition reaches the confirmation cycle threshold, the candidate real-scene anchor point is marked as confirmed and determined as the current real-scene anchor point. S43. Based on the position of the fixed real-scene feature corresponding to the current real-scene anchor point in the current real-scene image frame, the position of the fixed real-scene feature group in the anchor point coordinate system, and the content space correspondence, establish or update the current virtual-real space transformation relationship. The real-scene anchor point in the candidate state does not participate in the establishment or update of the current virtual-real space transformation relationship. S44. When the current real-scene anchor point fails to meet the anchor point confirmation condition within any verification cycle, it is changed from the confirmed state to the temporary holding state. The current virtual-real space conversion relationship is maintained according to the most recently confirmed virtual-real space conversion relationship and the terminal inertial motion increment during the temporary holding period. When the current real-scene anchor point meets the anchor point confirmation condition again within the anchor point holding time limit, it is restored to the confirmed state, and the current virtual-real space conversion relationship is updated according to the re-matched fixed real-scene features. S45. When the duration for which the current real-scene anchor point has not met the anchor point confirmation condition reaches the anchor point holding time limit, change its state from temporary holding to lost state, stop using the real-scene anchor point to update the current virtual-real space transformation relationship, and output the anchor point loss result.

6. The scenic area real-scene guide interaction method based on augmented reality and location awareness according to claim 5, characterized in that, In S5, the switching of the current real-scene anchor point is completed, including the following steps: S51. Determine whether the mobile terminal has entered the anchor point switching area associated with the current real-scene anchor point based on the current location of the mobile terminal. When entering the anchor point switching area, determine the adjacent real-scene anchor point corresponding to the current real-scene anchor point based on the scenic area real-scene anchor point reference data, and determine the adjacent real-scene anchor point as the real-scene anchor point to be switched. S52. While maintaining the current real-scene anchor point and its current virtual-real space conversion relationship, perform anchor point consistency verification on continuous real-scene image frames according to the fixed real-scene feature group corresponding to the real-scene anchor point to be switched, and manage the real-scene anchor point status of the real-scene anchor point to be switched in accordance with S4. S53. When the real-scene anchor point to be switched reaches the confirmed state, establish the virtual-real space conversion relationship to be switched based on the matching fixed real-scene features and content space correspondence corresponding to the real-scene anchor point to be switched. S54. Based on the anchor point spatial transformation relationship between the current real-scene anchor point and the real-scene anchor point to be switched, the current virtual-real spatial transformation relationship is transformed to the anchor point coordinates of the real-scene anchor point to be switched, forming a switching reference spatial transformation relationship. The positional transformation difference and direction transformation difference between the virtual-real spatial transformation relationship to be switched and the switching reference spatial transformation relationship are determined respectively. S55. When the position conversion difference and direction conversion difference do not exceed the corresponding switching continuity allowable amount, update the real-scene anchor point to be switched to the current real-scene anchor point, and update the virtual-real space conversion relationship to be switched to the current virtual-real space conversion relationship; when any conversion difference exceeds the corresponding switching continuity allowable amount, continue to maintain the current real-scene anchor point and the current virtual-real space conversion relationship before the switch; when the current real-scene anchor point before the switch becomes lost and the real-scene anchor point to be switched has not yet reached the confirmed state, return to S2 to re-form the current candidate anchor point set.

7. The interactive method for scenic area real-scene navigation based on augmented reality and location awareness according to claim 6, characterized in that, S5 also includes an anchor point recovery process based on near-field communication tags, the anchor point recovery process including the following steps: S56. When the current real-scene anchor point is lost and the mobile terminal reads the near-field communication tag set in the scenic area, obtain the tag identifier and tag reading time of the near-field communication tag. S57. Verify the label identifier according to the pre-stored correspondence between the label identifier, the label installation location and the real-scene anchor point identifier, and obtain the corresponding label installation location and target real-scene anchor point identifier after the label identifier passes the verification. S58. Based on the tag installation location, the positioning error range of the mobile terminal is redefined, and the target real-scene anchor point and the adjacent real-scene anchor points that intersect with the redefined positioning error range are determined as the set of recovery candidate anchor points. The target real-scene anchor point is marked as a candidate state instead of being directly marked as a confirmed state. S59. Obtain continuous real-scene image frames and terminal inertial motion data collected after the tag reading time, perform anchor point consistency verification on the target real-scene anchor point, when the target real-scene anchor point meets the anchor point confirmation condition, update it to the confirmed state and re-establish the current virtual-real space conversion relationship, when the target real-scene anchor point does not meet the anchor point confirmation condition, maintain its candidate state and do not trigger the corresponding augmented reality content interactive response.

8. The interactive method for scenic area real-view navigation based on augmented reality and location awareness according to claim 6, characterized in that, In S6, the augmented reality navigation interactive screen is generated and the interactive response is executed, including the following steps: S61. Based on the real-scene anchor point identifier of the current real-scene anchor point, retrieve the augmented reality content and content spatial correspondence associated with the current real-scene anchor point from the scenic area real-scene anchor point benchmark data. S62. When the current real-scene anchor point is in a confirmed state, the display position and display direction of the augmented reality content in the current real-scene image frame are determined by using the current virtual-real space transformation relationship; when the current real-scene anchor point is in a temporarily held state, the display position and display direction of the augmented reality content are determined by using the current virtual-real space transformation relationship maintained according to the terminal's inertial motion increment. S63. Superimpose augmented reality content onto the current real-world image frame according to the display position and display direction to form an augmented reality navigation interactive screen, and stop adding new interactive responses when the current real-world anchor point is lost. S64. Obtain the interactive input received by the mobile terminal, convert the screen position or terminal motion trajectory corresponding to the interactive input to the anchor coordinates of the current real-world anchor point, and determine whether the converted screen position or terminal motion trajectory enters the spatial interaction area corresponding to the augmented reality content. S65. When the current real-scene anchor point is in a confirmed state and the transformed screen position or terminal movement trajectory enters the spatial interaction area, execute the interactive response associated with the augmented reality content, update the interactive state of the augmented reality content, and pass the real-scene anchor point identifier, real-scene anchor point state, and current virtual-real space transformation relationship of the current real-scene anchor point to the next acquisition cycle.

9. A scenic area real-scene guide and interactive system based on augmented reality and location awareness, characterized in that, Includes a scenic area data server and a mobile terminal that communicates with the scenic area data server; The scenic area data server includes an anchor point data storage device and a server processor. The anchor point data storage device stores real-scene anchor point reference data of the scenic area. The real-scene anchor point reference data of the scenic area includes position constraint areas associated with multiple real-scene anchor points, allowed viewing directions, fixed real-scene feature groups, augmented reality content, content spatial correspondence, adjacent real-scene anchor points, anchor point switching areas, and anchor point spatial transformation relationships. The server processor retrieves the corresponding real-scene anchor point reference data of the scenic area from the anchor point data storage device and sends it to the mobile terminal according to the current location or real-scene anchor point identifier sent by the mobile terminal. The mobile terminal includes a location sensing component, an image acquisition component, an inertial sensing component, a display and interaction component, a terminal processor, and a near-field communication reading component connected to the terminal processor. The position sensing component acquires the current location of the mobile terminal and its positioning error range; the image acquisition component acquires continuous real-scene image frames; the inertial sensing component acquires terminal attitude data and terminal inertial motion data; and the terminal processor is connected to the position sensing component, the image acquisition component, the inertial sensing component, and the display interaction component, respectively. The terminal processor compares the positioning error range with the position constraint area of ​​each real-scene anchor point, and forms a current candidate anchor point set based on the correspondence between the shooting direction and the allowed observation direction determined by the terminal attitude data. It then matches the fixed real-scene feature groups of each candidate real-scene anchor point in the current candidate anchor point set with consecutive real-scene image frames, and forms an anchor point consistency verification result based on the cross-frame changes of the matched fixed real-scene features and the terminal inertial motion data. Based on the anchor point consistency verification result within the continuous verification period, the real-scene anchor points are managed as candidate, confirmed, temporarily maintained, or lost states. The current virtual-real space transformation relationship is established or updated only based on real-scene anchor points in the confirmed state, and in the temporarily maintained state, the current virtual-real space transformation relationship is maintained based on the most recently confirmed virtual-real space transformation relationship and the terminal inertial motion increment. After the mobile terminal enters the anchor point switching area, the terminal processor performs anchor point consistency verification on adjacent real-scene anchor points, determines the switching reference spatial conversion relationship of adjacent real-scene anchor points according to the anchor point spatial conversion relationship, and completes the switching of the current real-scene anchor point and the current virtual-real spatial conversion relationship when the adjacent real-scene anchor points reach the confirmed state and the corresponding spatial conversion difference does not exceed the allowable amount of switching continuity. The display interaction component overlays the augmented reality content associated with the current real-world anchor point onto continuous real-world image frames based on the current virtual-real space conversion relationship output by the terminal processor, and transmits the received interactive input to the terminal processor. The terminal processor executes the interactive response when the current real-world anchor point is in a confirmed state and the interactive input meets the spatial interaction conditions corresponding to the augmented reality content.

10. A scenic area real-scene guide interactive system based on augmented reality and location awareness according to claim 9, characterized in that, It also includes multiple near-field communication tags set up within the scenic area; each near-field communication tag is set up within the location constraint area or anchor point switching area of ​​the corresponding real-scene anchor point, and stores a tag identifier for distinguishing near-field communication tags. The anchor point data storage also stores the correspondence between the tag identifier, the tag installation location and the real-scene anchor point identifier. When the current real-scene anchor point is lost and the near-field communication reading component reads the tag identifier, the terminal processor sends the tag identifier to the scenic area data server. The server processor verifies the tag identifier according to the correspondence and returns the corresponding tag installation location and target real-scene anchor point identifier to the mobile terminal. The terminal processor re-limits the positioning error range according to the tag installation location, marks the target real-scene anchor point as a candidate state, and performs anchor point consistency verification on the target real-scene anchor point based on the continuous real-scene image frames collected after tag reading and the terminal inertial motion data. Only after the target real-scene anchor point reaches the confirmed state is the current virtual-real space conversion relationship re-established and the interactive response of the corresponding augmented reality content restored.

Images