Augmented reality navigation display method and device, mobile terminal and storage medium

Abstract

The application relates to the technical field of navigation display, and provides an augmented reality navigation display method and device, a mobile terminal and a storage medium. The method comprises the following steps: starting path guidance according to a destination, and automatically entering an augmented reality navigation mode when a user reaches a key node or the destination; in the augmented reality navigation mode, superimposedly displaying navigation guide information on a real scene picture captured by a camera; acquiring depth data and visual features of an environment where the mobile terminal is located; and dynamically adjusting a display mode of the navigation guide information on the real scene picture according to the depth data and the visual features. The application can dynamically adjust the display mode of the navigation guide information based on the depth data and the visual features, and ensures that the navigation guide information does not block real scene objects and interfere with user judgment.

Description

Technical Field

[0001] This application relates to the field of navigation display technology, and in particular to an augmented reality navigation display method, device, mobile terminal and storage medium. Background Technology

[0002] Traditional augmented reality navigation technologies, such as the scheme disclosed in CN107014393A, primarily rely on positioning data from the Global Positioning System (GPS) and inertial sensors, simply overlaying virtual navigation arrows or paths onto the real-time video stream captured by the mobile device's camera. While this approach provides an intuitive guidance method, the rendering of virtual navigation elements (such as arrows) is typically based on two-dimensional screen coordinates or simple three-dimensional spatial coordinates, lacking depth perception of the environment. This can lead to virtual navigation guidance inappropriately obscuring key real-world objects (such as traffic signs, pedestrians, or intersections), interfering with the user's judgment of the real environment. Summary of the Invention

[0003] In view of the above, it is necessary to propose an augmented reality navigation display method, mobile terminal and medium, which aims to solve the technical problem that the lack of environmental depth perception in traditional augmented reality navigation technology leads to improper virtual navigation guidance that obscures the real scene and interferes with the user's judgment.

[0004] A first aspect of this application provides an augmented reality navigation display method applied to a mobile terminal, the method comprising:

[0005] The system initiates route guidance based on the destination and automatically enters augmented reality navigation mode when the user reaches key nodes or the destination. In the augmented reality navigation mode, navigation guidance information is overlaid and displayed on the real-world image captured by the camera; Obtain depth data and visual features of the environment in which the mobile terminal is located; The display method of the navigation guidance information in the real-world scene is dynamically adjusted based on the depth data and visual features.

[0006] Optionally, dynamically adjusting the display method of the navigation guidance information in the real-world scene based on the depth data and visual features includes: The adjustment factor is determined based on the depth data and visual features; The fusion priority is determined based on the type of the navigation guidance information; The target rendering distance is obtained based on the initial spatial distance of the navigation guidance information, the fusion priority, and the adjustment factor; The display method of the navigation guidance information in the real-world scene is dynamically adjusted based on the target rendering distance.

[0007] Optionally, obtaining the target rendering distance based on the initial spatial distance of the navigation guidance information, the fusion priority, and the adjustment factor includes: Based on the initial spatial distance and the fusion priority, an initial weighted correction base is obtained; Based on the initial weighted correction base and the adjustment factor, the spatial distance correction value of the navigation guidance information is obtained; Based on the initial spatial distance and the spatial distance correction value, the target rendering distance of the navigation guidance information is obtained.

[0008] Optionally, dynamically adjusting the display method of the navigation guidance information in the real-world scene based on the target rendering distance includes: Determine the initial visual attributes of the navigation guidance information; The illumination intensity value and global contrast value are obtained based on the current real scene image, and the visual fusion coefficient is calculated based on the illumination intensity value and the global contrast value. The initial visual attributes are corrected using the visual fusion coefficient to obtain the target visual attributes for rendering; The navigation guidance information is rendered and displayed in the real-world scene based on the target rendering distance and the target visual attributes.

[0009] Optionally, the step of obtaining the illumination intensity value and global contrast value based on the current real-scene image, and calculating the visual fusion coefficient based on the illumination intensity value and the global contrast value includes: Based on the preset light intensity normalization reference value and the light intensity value, the light intensity normalization value is obtained; Based on the preset global contrast normalization reference value and the global contrast value, the global contrast normalization value is obtained. The visual fusion coefficient is obtained by weighting the normalized values ​​of illumination intensity and global contrast.

[0010] Optionally, the method further includes: When it is determined that the user has arrived at the destination, the system automatically switches from the augmented reality navigation mode to the geotagging and photo-taking mode. In the geotagging photography mode, framing guide icons are displayed on the live shooting screen; Save the environmental images containing destination features taken by the user based on the framing guide signs.

[0011] Optionally, the method further includes: Automatically associate metadata with the environmental image; The environmental images associated with the metadata are categorized and stored.

[0012] A second aspect of this application provides an augmented reality navigation display device, operating on a mobile terminal, the device comprising: The path guidance module is used to initiate path guidance based on the destination, and automatically enter augmented reality navigation mode when the user reaches a key node or the destination; A navigation overlay module is used to overlay navigation guidance information onto the real-world image captured by the camera in the augmented reality navigation mode; The data acquisition module is used to acquire depth data and visual features of the environment in which the mobile terminal is located; The display adjustment module is used to dynamically adjust the display method of the navigation guidance information in the real-world scene based on the depth data and visual features.

[0013] A third aspect of this application provides a mobile terminal, the mobile terminal including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement all or part of the steps of the augmented reality navigation display method.

[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements all or part of the steps of the augmented reality navigation display method.

[0015] The technical effects of this application are as follows: This application initiates path guidance based on the destination, automatically entering augmented reality navigation mode only when the user reaches key nodes or the destination and requires precise guidance, thus realizing on-demand activation of AR mode and avoiding redundant occlusion caused by overlaying virtual elements throughout the process; secondly, in AR mode, navigation guidance information is not simply overlaid, but rather the depth data and visual features of the mobile terminal's environment are first obtained, and the display method of navigation guidance information is dynamically adjusted based on the depth data and visual features to ensure that the navigation guidance information does not obscure real-world objects or interfere with the user's judgment. Attached Figure Description

[0016] Figure 1 A flowchart illustrating an augmented reality navigation display method provided in this application embodiment; Figure 2 A functional block diagram of an augmented reality navigation display device provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a mobile terminal provided in an embodiment of this application. Detailed Implementation

[0017] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. Unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing an embodiment in one alternative implementation and is not intended to be limiting of the application.

[0019] Example 1 Figure 1 This is a flowchart of an augmented reality navigation display method provided in an embodiment of this application. It is applied to a mobile terminal, which is a portable electronic device equipped with navigation functionality, such as a smartphone or a portable navigator. The augmented reality navigation display method includes the following steps.

[0020] S11, initiate path guidance based on the destination, and automatically enter augmented reality navigation mode when the user reaches a key node or the destination.

[0021] After receiving destination information entered by the user in the navigation application, the mobile terminal parses and verifies the validity of the destination information. If the destination information is valid, it uses the current location information and the verified destination information to generate the optimal driving route using a route planning algorithm, and automatically starts the route guidance function after the route planning is completed. If the destination information is invalid, it sends a prompt message to the user indicating that the destination cannot be recognized. The destination information includes at least one of the following: address text, point of interest (POI) name, and coordinate information.

[0022] As the user moves along the planned path, the mobile device continuously acquires its real-time location information and calculates the straight-line distance between the current location and the next key node or final destination. When this distance is less than a preset distance threshold (e.g., 20 meters for pedestrian navigation and 100 meters for in-vehicle navigation), the mobile device determines that the user has reached the key node or destination, that is, it determines that the user has reached a scene requiring augmented reality guidance. At this time, the mobile device activates its camera and switches the main interface of the navigation application to the real-world view captured by the camera, thus entering augmented reality navigation mode.

[0023] In this embodiment, the augmented reality navigation mode is only activated when the mobile terminal reaches a key node or destination, rather than throughout the entire navigation process. This is to save battery power, reduce computational load, and avoid interfering with the user on unnecessary road sections. Furthermore, the mobile terminal can automatically enter augmented reality navigation mode, achieving a smooth and automatic switch from traditional 2D map navigation to immersive augmented reality navigation without manual user operation, thus improving the smoothness and intelligence of the interaction.

[0024] S12, in the augmented reality navigation mode, navigation guidance information is overlaid on the real-world image captured by the camera.

[0025] Once augmented reality navigation mode is activated, the mobile device automatically wakes up the rear camera to capture real-time images of the surrounding environment. Navigation guidance information is then precisely overlaid onto the corresponding area of ​​the real-world view based on its spatial location. Users can access navigation guidance information directly by looking at the screen without switching interfaces, improving both the intuitiveness and safety of navigation.

[0026] The navigation guidance information includes at least one of the following: turning guidance information, lane guidance information, road condition guidance information, and key node guidance information. Turning guidance information includes directions such as left turn, right turn, straight ahead, U-turn, and roundabout driving, as well as corresponding distances; lane guidance information includes the location of the target lane and lane change prompts; road condition guidance includes speed limit prompts, congested road section prompts, and detour prompts for construction sections; key node guidance information includes the location and distance of highway entrances / exits, toll stations, and service areas.

[0027] For example, for turning guidance information, an arrow icon matching the driving direction is used and superimposed on the map at a preset distance in front of the corresponding turning point, with the remaining distance value marked; for lane guidance information, a semi-transparent lane outline graphic is used and superimposed on the lane area in front of the current vehicle position on the map, with the target driving lane marked with a highlight color; for road condition guidance information, text labels or icons are used and superimposed on the map at the location of the road segment where the corresponding road condition occurs, with the text labels parallel to the road direction on the map.

[0028] S13, acquire depth data and visual features of the environment in which the mobile terminal is located.

[0029] After the mobile terminal enters augmented reality navigation mode, it automatically wakes up the built-in RGB-D integrated sensor, which acquires depth data and visual features of the environment in real time. The RGB-D integrated sensor integrates an RGB imaging chip and a low-cost depth sensing chip, sharing the same optical path, and can acquire RGB images and depth data of the same scene at the same time.

[0030] In other embodiments, the mobile terminal may be equipped with an independent RGB camera and an independent TOF sensor, which can acquire RGB images through the RGB camera and acquire depth data through the TOF sensor.

[0031] Visual features are extracted from RGB images, describing environmental optical properties, including at least one of brightness, contrast, and color distribution. Depth data refers to the distance information from each point in the scene to the camera. The extracted visual features and depth data can be linked to the same timestamp.

[0032] S14, dynamically adjust the display method of the navigation guidance information in the real-world scene based on the depth data and visual features.

[0033] If navigation guidance information uses fixed display parameters (brightness, size, transparency, etc.), its readability will be insufficient due to real-time changes in the mobile terminal's environment (different brightness, occlusion, scene complexity), making it easy for users to miss or misread it, thus affecting navigation accuracy and safety. By automatically adjusting the display method of navigation guidance information in the real-world scene based on real-time acquired depth data and visual features, precise adaptation between the environment and the guidance can be achieved, ensuring that the navigation guidance information is always clear and eye-catching, reducing the risk of deviations and violations caused by unclear guidance.

[0034] In an optional embodiment, dynamically adjusting the display method of the navigation guidance information in the real-world scene based on the depth data and visual features includes: The adjustment factor is determined based on the depth data and visual features; The fusion priority is determined based on the type of the navigation guidance information; The target rendering distance is obtained based on the initial spatial distance of the navigation guidance information, the fusion priority, and the adjustment factor; The display method of the navigation guidance information in the real-world scene is dynamically adjusted based on the target rendering distance.

[0035] The mobile terminal pre-stores a mapping rule table, which records the correspondence between depth data, visual features (brightness, contrast), and adjustment factors. Using this mapping rule table, a dynamic adjustment factor for correcting the navigation guidance spatial distance can be determined based on the acquired depth data and visual features (brightness, contrast). For example, low depth data, low light, and low contrast correspond to a large adjustment factor; high depth data, strong light, and high contrast correspond to a small adjustment factor, providing a quantitative basis for correcting the navigation guidance spatial distance. The adjustment factor is a dynamic correction coefficient determined based on real-time acquired environmental depth data and visual features. Its value is determined by the current environmental adaptation requirements and is used to quantify the correction magnitude of the initial spatial distance, achieving on-demand adaptation of the guidance distance.

[0036] In AR navigation mode, each navigation guidance message is treated as an independent rendering object, and each object is configured with a unique fusion priority and initial spatial distance. The initial spatial distance of the navigation guidance message refers to the initial spatial presentation distance of the navigation guidance message in the navigation display interface, based on the original map scale and the planned navigation path, and adapted to the current display viewpoint of the mobile terminal. It represents the initial relative spatial position of the guidance message and the mobile terminal's display reference point. In other words, the initial spatial distance refers to the initial depth of the object from the camera.

[0037] Fusion priority refers to a weighted parameter determined based on the type of navigation guidance information (such as turn guidance, lane guidance, etc.), used to distinguish the adjustment priority of different types of navigation guidance information. The value range of fusion priority is [0,1), and its magnitude is related to the importance of navigation guidance information and the degree of avoidance of real objects. The higher the fusion priority value, the lower the priority and importance of the navigation guidance information, and the higher the degree of avoidance of real objects, giving priority to avoiding occlusion of real objects; the lower the fusion priority value, the higher the priority and importance of the navigation guidance information, and the lower the degree of avoidance of real objects, with no need for priority avoidance. For example, for core navigation arrows pointing to key turns or highway entrances / exits, their fusion priority can be set to 0.1, the highest priority, requiring no avoidance of real objects, adjusting their depth to the closest, and rendering them in the foreground (rendered in a closer position), making them visually the most prominent; for regular straight-ahead arrows, their fusion priority can be set to 0.3, the second highest priority, appropriately avoiding real objects, adjusting their depth to medium, and rendering them in the middle layer; for information labels such as road names and traffic condition notes, the fusion priority can be set to 0.5, the lowest priority, prioritizing avoidance of real objects, adjusting their depth to a farther position, and rendering them in the second background layer; for navigation interface background information and decorative auxiliary elements, the fusion priority can be set to 0.8, the lowest priority, completely avoiding real objects, adjusting their depth to the farthest position, and relegating them to the background layer (rendered in a farther position), without interfering with the recognition of core guidance.

[0038] The initial spatial distance of the navigation guidance information is the initial spatial presentation distance set for the navigation guidance information in the navigation display interface. Depth data is used to understand the spatial layout of the scene, providing a basis for the subsequent reasonable placement of navigation guidance information and avoiding spatial conflicts. In augmented reality navigation scenarios, spatial distance is defined as the relative distance from the camera to the virtual navigation guidance object. The larger the distance value, the farther the virtual object appears in the real scene; the smaller the distance value, the closer the virtual object appears in the real scene. This application embodiment uses the preset initial spatial distance of the navigation guidance information as a benchmark, and calculates the optimal target rendering distance of the navigation guidance information in the real scene through dual optimization of priority weighted assignment and dynamic correction of adjustment factors. The target rendering distance, that is, the optimal spatial presentation distance of the navigation guidance information in the real scene after dynamic correction, can accurately match the current environment adaptation requirements and user visual perception habits, ensuring that the navigation guidance information is clear and distinguishable without obscuring key information in the real scene, and taking into account both recognizability and the integrity of the real scene information.

[0039] Finally, the display method of the navigation guidance information in the camera's real-world view is dynamically adjusted based on the target rendering distance. The display method may include, but is not limited to: display size, floating position, and overlay layer. When the distance is close, the information is enlarged and placed at the top; when the distance is far, it is reduced to fit the view, ensuring that the navigation guidance information is both accurate to the real-world view and clearly visible.

[0040] In an optional embodiment, obtaining the target rendering distance based on the initial spatial distance of the navigation guidance information, the fusion priority, and the adjustment factor includes: Based on the initial spatial distance and the fusion priority, an initial weighted correction base is obtained; Based on the initial weighted correction base and the adjustment factor, the spatial distance correction value of the navigation guidance information is obtained; Based on the initial spatial distance and the spatial distance correction value, the target rendering distance of the navigation guidance information is obtained.

[0041] Based on the initial spatial distance of the navigation guidance information, and combined with the fusion priority corresponding to the guidance type, the initial weighted correction base is obtained through multiplication. The purpose is to assign a weighting coefficient to the initial spatial distance according to the importance of the guidance. The higher the priority, the closer the initial weighted correction base is to the core display requirements, thus determining the basis for subsequent corrections.

[0042] Based on the initial weighted correction base obtained in the first step, and combined with the adjustment factor determined based on environmental depth data and visual features, a spatial distance correction value is obtained through multiplication. This spatial distance correction value is used to quantify the magnitude of the initial spatial distance adjustment, adapting to the visual perception requirements of the current environment. The spatial distance correction value changes synchronously with different environments.

[0043] Using the initial spatial distance of the navigation guidance information as a baseline, the spatial distance correction value obtained in the second step is added together, and the final target rendering distance is obtained. If the spatial distance correction value is positive, the initial spatial distance is increased; if the spatial distance correction value is negative, the initial spatial distance is decreased. This ensures that the guidance is presented at a distance that is appropriate to the current environment and its own importance in the real scene, taking into account both prominence and avoidance requirements.

[0044] Optionally, the formula for calculating the target rendering distance can be expressed as d1=d+β (d0) w), where d0 represents the initial spatial distance, w represents the fusion priority, β represents the adjustment factor, and d1 represents the target rendering distance.

[0045] First, the initial weighted correction base is obtained by weighted calculation based on the fusion priority. Then, the initial weighted correction base is corrected by the adjustment factor to obtain the spatial distance correction value. This spatial distance correction value is the offset applied to the initial spatial distance, and the offset is proportional to the initial spatial distance and the fusion priority.

[0046] Where, β The operation logic of d ensures that the offset is proportional to the scene scale, making the adjustment more visually natural. β is used to control the overall adjustment range. In this way, it can reduce the occlusion of navigation guidance information on key real-world scenes, optimize the sense of spatial hierarchy, and make navigation guidance blend more naturally into the real environment.

[0047] As can be seen, when the fusion priority is 0 (highest priority), the offset is 0, the target rendering distance equals the initial spatial distance, and the navigation guidance information is precisely placed in its theoretical position, ensuring the core guidance stands out visually. When the fusion priority > 0, the offset is positive, the target rendering distance is greater than the initial spatial distance, meaning that low-priority navigation guidance information will be pushed away, effectively avoiding occlusion of real objects. For example, assuming the navigation guidance information is a straight arrow with a fusion priority of 0.3, if the theoretical position of this arrow would pass through a street lamp in the real scene, the above calculation of the target rendering distance can ensure that the straight arrow is rendered behind the street lamp, thus avoiding occlusion. This mechanism significantly reduces the occlusion of key real-world elements by navigation guidance information, optimizes the spatial hierarchy of the AR image, and allows navigation guidance information to blend more naturally into the real environment, balancing guidance recognition and real-world integrity.

[0048] After obtaining the optimal target rendering distance from the navigation guidance information, the rendering engine will precisely place the navigation guidance information at a distance of d1 along the line of sight from the camera to the target point P.

[0049] For example, assuming depth data shows the mobile terminal is 80 meters from the right-turn intersection (a complex close-up scene), with visual characteristics of current light intensity L=20 lux (low light scene) and contrast C=30 (low contrast), using the mapping rule table, the adjustment factor corresponding to close distance, low light, and low contrast is 0.6; the current navigation guidance information is a right-turn intersection guidance, which is a key node guidance. Based on the preset guidance type-fusion priority mapping rule, the fusion priority of the right-turn intersection guidance is determined to be 0.9; the initial spatial distance of this right-turn intersection guidance is 5cm, so the final target rendering distance is 7.7cm; based on the target rendering distance, the mobile terminal can adjust the display method of the right-turn intersection guidance in the camera's real-world view: enlarging the guidance arrow from the initial 5cm size to 7.7cm, placing it on top of the unobstructed area on the right side of the screen, increasing the display brightness by 40%, enhancing the contrast by 30%, and ensuring the guidance accurately matches the real-world intersection scene, allowing the user to clearly identify it by looking directly at the screen without manual adjustment.

[0050] For example, assuming depth data shows that the mobile terminal is 600 meters away from the straight-ahead guidance point (long distance), and the visual characteristics are current light intensity L=180 lux (strong light) and contrast ratio C=120 (high contrast), using the mapping rule table, the adjustment factor corresponding to long distance, strong light, and high contrast is -0.3; the current navigation guidance information is a straight-ahead guidance, which is a normal guidance, and based on the preset guidance type-fusion priority mapping rule, the fusion priority of the straight-ahead guidance is determined to be 0.5; the initial spatial distance of this straight-ahead guidance is 6cm, so the final target rendering distance is 5.1cm; based on the target rendering distance, the mobile terminal can adjust the display method of the straight-ahead guidance in the camera's real-world view: reduce the size of the straight-ahead guidance from 6cm to 5.1cm, reduce the display brightness by 25%, place it at the bottom of the screen, so as not to obstruct the real-world road, and adapt to the visual needs of strong light long-distance scenes.

[0051] The above optional embodiments, by calculating the target rendering distance, enable navigation guidance information to intelligently adapt to the depth of the environment, automatically avoid occlusion of key real-world information, and improve the spatial rationality of AR guidance and user experience.

[0052] In an optional embodiment, dynamically adjusting the display method of the navigation guidance information in the real-world scene based on the target rendering distance includes: Determine the initial visual attributes of the navigation guidance information; The illumination intensity value and global contrast value are obtained based on the current real scene image, and the visual fusion coefficient is calculated based on the illumination intensity value and the global contrast value. The initial visual attributes are corrected using the visual fusion coefficient to obtain the target visual attributes for rendering; The navigation guidance information is rendered and displayed in the real-world scene based on the target rendering distance and the target visual attributes.

[0053] Initial visual attributes are a set of preset visual presentation parameters for navigation guidance information under pre-defined standard environmental conditions (i.e., light intensity of 500 lux ± 50 lux, global contrast ratio of 80 ± 10, simulating a typical scene of a sunny midday without obstructions), to ensure basic visibility and integration. Initial visual attributes may include one or more of the following: brightness parameters, contrast parameters, color parameters, and transparency parameters. Brightness parameters determine the lightness or darkness of the navigation guidance information in the image; contrast parameters refer to the degree of difference between bright and dark areas within the navigation guidance information itself, reflecting the sharpness of the edges of the navigation guidance information; color parameters include the primary color tone (such as red for turn arrows, green for straight-line guidance), color saturation, and color temperature. Transparency parameters refer to the transparency of the navigation guidance information, used to ensure that the guidance information is clearly visible without completely obscuring the actual scene below.

[0054] Based on extensive user visual perception experimental data and augmented reality navigation scenario adaptation requirements, the initial visual attributes of various navigation guidance information can be obtained through statistical analysis and optimization. For example, the initial brightness of core navigation guidance information such as key turning arrows and highway entrance / exit signs can be set to 180 cd / m² (higher than regular guidance) to enhance visual prominence; while the initial brightness of secondary navigation guidance information such as road name labels and background decorative elements can be set to 120 cd / m² with a transparency of 0.8 to reduce visual interference.

[0055] The visual fusion coefficient is a predefined scalar parameter with a value ranging from [0, 2]. Its value reflects the comprehensive impact of the current environment on the visibility and integration friendliness of navigation guidance information. The visual fusion coefficient provides a quantitative basis for the correction of initial visual attributes, achieving a precise match between the adjustment range of visual attributes and the complexity of the environment. A higher visual fusion coefficient usually indicates more complex environmental visual conditions (such as strong light, high contrast, or special color temperature scenes), requiring more significant adjustments to the initial visual attributes to ensure clear visibility and harmonious integration of navigation guidance information; a lower visual fusion coefficient indicates more favorable environmental visual conditions (such as normal sunny days or bright indoor scenes), requiring less adjustment to the initial visual attributes to maintain good visibility and integration. For example, the visual fusion coefficient for a typical sunny, unobstructed scene (illuminance 500 lux, global contrast 85) is 0.5, requiring no significant adjustment to the initial visual attributes; the visual fusion coefficient for a sunny midday bright light scene (illuminance 1200 lux, global contrast 150) is 1.8, requiring a significant increase in the contrast and edge sharpness of the guidance information, while appropriately reducing brightness to avoid glare; the visual fusion coefficient for a nighttime low-light scene (illuminance 30 lux, global contrast 30) is 1.2, requiring an increase in the brightness of the guidance information, while adjusting the color temperature to a warm tone (3000K) to avoid eye strain.

[0056] The initial visual attributes are corrected using a visual fusion coefficient, with the correction magnitude increasing as the visual fusion coefficient increases and decreasing as the visual fusion coefficient decreases. Optionally, the target visual attribute can be obtained by calculating the product of the visual fusion coefficient and the initial visual attribute.

[0057] The target rendering distance and target visual attributes are integrated. The navigation guidance information is superimposed on the real-time scene captured by the camera through the augmented reality rendering engine built into the mobile terminal. This achieves dual optimization of spatial location adaptation and visual presentation adaptation, ensuring that the navigation guidance information is not only harmoniously integrated with the real scene in space to avoid obscuring key real-scene objects, but also clearly visible and visually comfortable and natural.

[0058] Augmented reality rendering engines possess spatial positioning, layer overlay, and real-time rendering capabilities. They can determine the three-dimensional spatial coordinates of navigation guidance information in the real-world scene based on the target rendering distance, and adjust the rendering parameters of the guidance information according to the target's visual attributes. Finally, the guidance information is overlaid onto the corresponding position in the real-world scene to form a complete AR navigation screen.

[0059] The above-mentioned optional embodiments calculate the visual fusion coefficient by sensing the ambient light intensity and global contrast value in real time, ensuring that the navigation guidance information has clear visibility in various complex environments. This solves the problem that guidance information with fixed visual attributes is easily submerged, glaring, or blurry in extreme environments. At the same time, combined with spatial adaptation of the target rendering distance, the navigation guidance information avoids obscuring key real-world objects in space and blends harmoniously with the real-world scene visually, reducing the sense of visual disjointedness and improving the user experience of AR navigation.

[0060] In an optional embodiment, the step of obtaining the illumination intensity value and global contrast value based on the current real-scene image, and calculating the visual fusion coefficient based on the illumination intensity value and the global contrast value, includes: Based on the preset light intensity normalization reference value and the light intensity value, the light intensity normalization value is obtained; Based on the preset global contrast normalization reference value and the global contrast value, the global contrast normalization value is obtained. The visual fusion coefficient is obtained by weighting the normalized values ​​of illumination intensity and global contrast.

[0061] The normalized reference value for illumination intensity, L_max, is a preset threshold for the maximum illumination intensity that may occur in the navigation scenario. A linear normalization algorithm can be used to normalize the illumination intensity value based on the reference value, yielding the normalized illumination intensity value. The calculation formula is: L_norm = L / L_max, where L is the real-time extracted illumination intensity value, and L_norm is the normalized illumination intensity value, ranging from [0, 1].

[0062] The global contrast normalization reference value C_max is a preset maximum global contrast threshold that may occur in a navigation scenario. A linear normalization algorithm can be used to normalize the global contrast value based on the global contrast normalization reference value to obtain the global contrast normalization value. The calculation formula is: C_norm = C / C_max, where C is the real-time extracted global contrast value, and C_norm is the global contrast normalization value, with a value range of [0, 1].

[0063] Based on the weighted impact of illumination intensity and global contrast on navigation guidance visibility, different weighting coefficients are assigned to achieve comprehensive quantification of the two types of features. The sum of the weighting coefficients for illumination intensity and global contrast is 1. A visual fusion coefficient is obtained by weighting the normalized value of illumination intensity, the weighting coefficient of illumination intensity, the normalized value of global contrast, and the weighting coefficient of global contrast.

[0064] By normalizing the light intensity values ​​and global contrast of different magnitudes to the same range, the influence of dimensional differences on subsequent calculations can be eliminated.

[0065] To further improve the ability of the visual fusion coefficient to distinguish the complexity of the environment, the calculation results can be linearly amplified. For example, the visual fusion coefficient can be linearly amplified by 2 times so that the value range of the amplified visual fusion coefficient is [0, 2].

[0066] Furthermore, since low-light nighttime scenes (e.g., L≤100 lux) are complex visual environments requiring a higher visual fusion coefficient, it is necessary to determine whether the visual fusion coefficient is greater than a preset threshold for low-light nighttime scenes. If the visual fusion coefficient is less than the preset threshold, an inverse linear normalization algorithm is used to normalize the illumination intensity value based on a normalized reference value, resulting in a normalized illumination intensity value. The calculation formula is: L_norm = 1 - (L / L_max).

[0067] In an optional embodiment, the method further includes: When it is determined that the user has arrived at the destination, the system automatically switches from the augmented reality navigation mode to the geotagging and photo-taking mode. In the geotagging photography mode, framing guide icons are displayed on the live shooting screen; Save the environmental images containing destination features taken by the user based on the framing guide signs.

[0068] When the mobile device determines that the user has arrived at the destination, it automatically switches from augmented reality navigation mode to geotagging photo mode. In geotagging photo mode, the interface changes to a viewfinder-centered photo interface and displays framing guidance icons to guide the user to take one or more environmental images containing destination features and save the environmental images.

[0069] In the geotagging photography mode, the mobile terminal automatically utilizes its built-in orientation sensors (including a gyroscope, magnetometer, and accelerometer) to collect real-time attitude data (including pitch, roll, and yaw angles) and geomagnetic orientation information of its location. Simultaneously, the mobile terminal combines its current coordinates to calculate the azimuth angle pointing from its current location to the destination, thereby generating an optimal shooting orientation suggestion. This optimal shooting orientation ensures that the destination is centered in the captured image and that the shooting angle avoids obstructions, guaranteeing accurate geotagging of the captured image with respect to the destination.

[0070] To clearly guide users in adjusting their shooting orientation, the mobile device overlays a framing guide icon (AR arrow) onto the camera's live preview screen. This AR arrow is rendered based on the spatial coordinates of the real-world scene, superimposed on the direction requiring adjustment, and its position and angle update in real time as the user's mobile device posture changes. Following the guidance of the framing guide icon, the user gradually adjusts the mobile device's camera orientation. When the deviation between the mobile device's posture data and the optimal shooting orientation is less than a preset threshold (e.g., 3°), the system informs the user that they are at the optimal shooting orientation through screen highlighting, vibration feedback, or voice prompts. At this point, the user can trigger the photo capture command to take a picture of the surrounding environment.

[0071] In existing technologies, augmented reality navigation and geographic information recording are two separate functional modules. Users need to manually switch between these modules from the end of navigation to the start of geographic information recording, resulting in an incoherent process. Furthermore, the recorded information (images) lacks deep connection with the navigation context, limiting its application value in subsequent navigation (such as as visual landmarks) or information sharing. However, the embodiments of this application automatically and intelligently guide users to complete high-quality geographic information recording at the navigation endpoint, solving the problem of the separation between augmented reality navigation and geographic information recording.

[0072] In an optional embodiment, the method further includes: Automatically associate metadata with the environmental image; The environmental images associated with the metadata are categorized and stored.

[0073] After users manually or automatically capture environmental images, these images are stored in a geotagged album or a structured database. A set of metadata can be automatically generated and embedded for each environmental image. This metadata may include one or more of the following: destination geographic coordinates, shooting orientation (azimuth and pitch angles), depth data, visual features, fusion priority, visual fusion coefficients, navigation path ID, and timestamp. After embedding the metadata, the environmental images are automatically tagged or categorized based on the metadata.

[0074] Metadata records the environmental optical conditions at the time of shooting, enabling semantic storage of geotagged data. Automatic classification based on metadata allows users to easily filter and search historical photos by environmental conditions, providing data support for subsequent intelligent applications.

[0075] The destination's geographic coordinates, recommended navigation routes, and environmental images can be sent to other users' terminal devices via third-party applications, expanding the functional boundaries of traditional navigation.

[0076] Furthermore, the display of environmental images can be adjusted based on the visual fusion coefficients in the metadata to make them more immersive and reduce jarring effects.

[0077] For example, suppose that during a future navigation session, when user U1 arrives at a location whose geographic information has been historically recorded, the system can automatically trigger a comparison function. For instance, the system retrieves historical environmental images and their metadata stored locally or in the cloud for that location. Simultaneously, the system obtains the visual fusion coefficient k_now of the current real-time environment and the visual fusion coefficient k_old corresponding to the historical environmental image, and calculates the target transparency T = T_base. (k_now / k_old), where T_base is the preset baseline transparency. If the current environment is nighttime and the historical environment image was taken during the day, then k_now > k_old, and the target transparency T will increase. This means that the historical environment image will be overlaid with higher transparency (lighter) to avoid the overly bright daytime image from impairing night vision capabilities.

[0078] The historical environment image, adaptively adjusted, is overlaid onto the current real-time video feed as an augmented reality layer based on the saved pose and coordinates. Users can see a semi-transparent historical imprint blended with the current reality. Even if the appearance of objects changes due to time, season, or lighting, the target can be intuitively identified through visual comparison, greatly improving the accuracy and enjoyment of navigation.

[0079] Since changes in lighting conditions can cause visual conflicts, directly overlaying historical environmental images can result in poor performance. This application's embodiments introduce two quantization parameters, k_now and k_old, for visual fusion, which intelligently adjust the display state of historical environmental images to adapt their visual performance to current environmental conditions, achieving effective cross-time visual comparison.

[0080] Example 2 Figure 2 This is a functional block diagram of the augmented reality navigation display device provided in the embodiments of this application.

[0081] In some embodiments, the augmented reality navigation display device 20 may include multiple functional modules composed of program code segments. The program code of each program segment in the augmented reality navigation display device 20 may be stored in the memory of the mobile terminal and executed by at least one processor to perform (see details). Figure 1 (Description) Augmented reality navigation display functionality.

[0082] In this embodiment, the augmented reality navigation display device 20 can be divided into multiple functional modules according to its functions. These functional modules may include: a path guidance module 201, a navigation overlay module 202, a data acquisition module 203, a display adjustment module 204, an image capture module 205, and a classification and storage module 206. The term "module" in this application refers to a series of computer-readable instruction segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0083] The path guidance module 201 is used to initiate path guidance according to the destination, and automatically enter the augmented reality navigation mode when the user reaches the key node or the destination. The navigation overlay module 202 is used to overlay navigation guidance information onto the real-world image captured by the camera in the augmented reality navigation mode. The data acquisition module 203 is used to acquire depth data and visual features of the environment in which the mobile terminal is located; The display adjustment module 204 is used to dynamically adjust the display method of the navigation guidance information in the real scene based on the depth data and visual features; The image capturing module 205 is used to automatically switch from the augmented reality navigation mode to the geotagging shooting mode when it is determined that the user has arrived at the destination; in the geotagging shooting mode, it displays a framing guide mark on the real-time shooting screen; and saves the environmental image containing the destination features captured by the user according to the framing guide mark. The classification and storage module 206 is used to automatically associate metadata with the environmental image and to classify and store the environmental images associated with the metadata.

[0084] It should be understood that the various variations and specific embodiments of the augmented reality navigation display method provided in the above embodiments are also applicable to the augmented reality navigation display device in this embodiment. Through the detailed description of the augmented reality navigation display method described above, those skilled in the art can clearly understand the implementation process of the augmented reality navigation display device in this embodiment. For the sake of brevity, it will not be described in detail here.

[0085] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the above-described augmented reality navigation display method embodiment.

[0086] Example 4 See Figure 3The diagram shown is a structural schematic of a mobile terminal provided in an embodiment of this application. In a preferred embodiment of this application, the mobile terminal 3 includes: a memory 301, at least one processor 302, at least one communication bus 303, multiple sensors 304, and a display screen 305.

[0087] In some embodiments, the plurality of sensors 304 may include an optical camera, a depth sensor, an inertial measurement unit (IMU), a magnetometer, and GPS. The optical camera is used to capture a color image (RGB image) of the environment. The depth sensor, which may employ structured light, time-of-flight (ToF), or binocular vision, is used to acquire a depth map of the scene. The value of each pixel in the depth map represents the physical distance of that point from the sensor. The inertial measurement unit (IMU) includes an accelerometer and a gyroscope for sensing the device's motion, attitude, and acceleration. The magnetometer is used to sense the device's orientation relative to the Earth's magnetic north. GPS is used to obtain a coarse geographical location of the device.

[0088] Those skilled in the art should understand that Figure 3 The structure of the mobile terminal shown does not constitute a limitation of the embodiments of this application. The mobile terminal 3 may also include more or fewer other hardware or software, or different component arrangements than shown.

[0089] In some embodiments, the memory 301 stores a computer program and an operating system, which, when executed by the at least one processor 302, implements all or part of the steps in the augmented reality navigation display method described above. The memory 301 includes read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data. Further, the computer-readable storage medium may primarily include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for a function, etc.

[0090] In some embodiments, the at least one processor 302 is the control unit of the mobile terminal 3, connecting various components of the mobile terminal 3 via various interfaces and lines. It executes programs or modules stored in the memory 301 and calls data stored in the memory 301 to perform various functions and process data of the mobile terminal 3. For example, when the at least one processor 302 executes a computer program stored in the memory, it implements all or part of the steps of the augmented reality navigation display method described in this application embodiment; or it implements all or part of the functions of the augmented reality navigation display device. The at least one processor 302 may be composed of integrated circuits, such as a single-packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0091] In some embodiments, the at least one communication bus 303 is configured to enable communication between the memory 301 and the at least one processor 302, etc. Although not shown, the mobile terminal 3 may also include a power supply (e.g., a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 302 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power supplies, a rechargeable power fault detection circuit, a power converter or inverter, a power status indicator, or any other components.

[0092] The mobile terminal 3 may also include a Bluetooth module, a Wi-Fi module, internal memory, a network interface, an input location, and a display screen, etc., which will not be described in detail here.

[0093] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a mobile terminal to execute portions of the methods described in the various embodiments of this application.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0095] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. An augmented reality navigation display method, applied to a mobile terminal, characterized in that, The method includes: The system initiates route guidance based on the destination and automatically enters augmented reality navigation mode when the user reaches key nodes or the destination. In the augmented reality navigation mode, navigation guidance information is overlaid and displayed on the real-world image captured by the camera; Obtain depth data and visual features of the environment in which the mobile terminal is located; The display method of the navigation guidance information in the real-world scene is dynamically adjusted based on the depth data and visual features.

2. The augmented reality navigation display method according to claim 1, characterized in that, The step of dynamically adjusting the display method of the navigation guidance information in the real-world scene based on the depth data and visual features includes: The adjustment factor is determined based on the depth data and visual features; The fusion priority is determined based on the type of the navigation guidance information; The target rendering distance is obtained based on the initial spatial distance of the navigation guidance information, the fusion priority, and the adjustment factor; The display method of the navigation guidance information in the real-world scene is dynamically adjusted based on the target rendering distance.

3. The augmented reality navigation display method according to claim 2, characterized in that, The process of obtaining the target rendering distance based on the initial spatial distance of the navigation guidance information, the fusion priority, and the adjustment factor includes: Based on the initial spatial distance and the fusion priority, an initial weighted correction base is obtained; Based on the initial weighted correction base and the adjustment factor, the spatial distance correction value of the navigation guidance information is obtained; Based on the initial spatial distance and the spatial distance correction value, the target rendering distance of the navigation guidance information is obtained.

4. The augmented reality navigation display method according to claim 2, characterized in that, The method of dynamically adjusting the display of the navigation guidance information in the real-world scene based on the target rendering distance includes: Determine the initial visual attributes of the navigation guidance information; The illumination intensity value and global contrast value are obtained based on the current real scene image, and the visual fusion coefficient is calculated based on the illumination intensity value and the global contrast value. The initial visual attributes are corrected using the visual fusion coefficient to obtain the target visual attributes for rendering; The navigation guidance information is rendered and displayed in the real-world scene based on the target rendering distance and the target visual attributes.

5. The augmented reality navigation display method according to claim 4, characterized in that, The process of obtaining the illumination intensity value and global contrast value based on the current real-world image, and calculating the visual fusion coefficient based on the illumination intensity value and the global contrast value, includes: Based on the preset light intensity normalization reference value and the light intensity value, the light intensity normalization value is obtained; Based on the preset global contrast normalization reference value and the global contrast value, the global contrast normalization value is obtained. The visual fusion coefficient is obtained by weighting the normalized values ​​of illumination intensity and global contrast.

6. The augmented reality navigation display method according to claim 1, characterized in that, The method further includes: When it is determined that the user has arrived at the destination, the system automatically switches from the augmented reality navigation mode to the geotagging and photo-taking mode. In the geotagging photography mode, framing guide icons are displayed on the live shooting screen; Save the environmental images containing destination features taken by the user based on the framing guide signs.

7. The augmented reality navigation display method according to claim 6, characterized in that, The method further includes: Automatically associate metadata with the environmental image; The environmental images associated with the metadata are categorized and stored.

8. An augmented reality navigation display device, operating on a mobile terminal, characterized in that, The device includes: The path guidance module is used to initiate path guidance based on the destination, and automatically enter augmented reality navigation mode when the user reaches a key node or the destination; A navigation overlay module is used to overlay navigation guidance information onto the real-world image captured by the camera in the augmented reality navigation mode; The data acquisition module is used to acquire depth data and visual features of the environment in which the mobile terminal is located; The display adjustment module is used to dynamically adjust the display method of the navigation guidance information in the real-world scene based on the depth data and visual features.

9. A mobile terminal, characterized in that, The mobile terminal includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements all or part of the steps of the augmented reality navigation display method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements all or part of the steps of the augmented reality navigation display method according to any one of claims 1 to 7.

Images