Navigation method and navigation device of augmented reality equipment, equipment and storage medium

Abstract

The invention discloses a navigation method and a navigation device for augmented reality equipment, the equipment and a storage medium. The navigation method comprises the following steps: acquiring an environment image of augmented reality equipment in a preset range of a current position; identifying a navigation reference object in the environment image; acquiring preset content of the navigation reference object; and converting the preset content into a navigation identifier for display, wherein the navigation identifier and the navigation reference object are in an anchoring state. According to the method, the displayed navigation identifier is anchored in the environment, navigation guidance is provided for the user by using the navigation identifier, the navigation identifier and the environment can be effectively fused, and the guidance and intuition of the navigation process are improved.

Description

Technical Field

[0001] The present invention belongs to the field of navigation technology, and in particular, relates to a navigation method of an augmented reality device, a navigation apparatus, a computer device and a computer-readable storage medium. Background Art

[0002] Extended reality technologies such as virtual reality (VR), augmented reality (AR), and mixed reality (MR) are computer simulation technologies that can create and experience a virtual world. They use computers to generate a simulated environment, which is a multi-source information fusion, interactive three-dimensional dynamic vision, and system simulation of entity behavior, allowing users to immerse themselves in the environment.

[0003] With the development of AR and other technologies, AR technology is used in more and more scenarios. Among them, map navigation is a common scenario in people's daily life. If AR technology can be applied to map navigation, the combination of virtual and real characteristics of AR technology will be used to improve the navigation experience.

[0004] Therefore, there is an urgent need to provide a solution that can integrate navigation with the real environment. Summary of the invention

[0005] The technical problem solved by the present invention is: how to improve the integration degree of navigation icons and real environment during navigation of augmented reality device, so as to enhance the guidance and intuitiveness of navigation.

[0006] The present application discloses a navigation method for an augmented reality device, the navigation method comprising:

[0007] Acquire an environment image of the augmented reality device within a predetermined range of the current position;

[0008] identifying a navigation reference object in the environment image;

[0009] Acquire preset content of the navigation reference object;

[0010] The preset content is converted into a navigation mark for display, and the navigation mark and the navigation reference object are in an anchored state.

[0011] Optionally, the navigation method further includes determining a current position of the augmented reality device, wherein determining the current position of the augmented reality device includes:

[0012] Using the augmented reality device to capture an image within a preset range of the augmented reality device;

[0013] Identifying a reference object in an image within a preset range of the augmented reality device and acquiring a geographic location of the reference object;

[0014] Calculating the relative distance between the augmented reality device and the reference object;

[0015] The current position of the augmented reality device is determined according to the relative distance and the geographical position of the reference object.

[0016] Optionally, acquiring the geographical location of the reference object includes:

[0017] Obtaining the geographic location of the augmented reality device according to a satellite positioning method;

[0018] Acquiring local area map data covering the geographical location;

[0019] The reference object is matched in the local map data to obtain the geographical location of the reference object.

[0020] Optionally, the navigation method further includes:

[0021] Detecting the light intensity of the environment image;

[0022] The display parameters when the preset content is converted into a navigation mark for display are adjusted according to the light intensity.

[0023] Optionally, the navigation method further includes:

[0024] Detecting the relative spatial position of the augmented reality device and the navigation reference object;

[0025] The display position and / or display size of the navigation mark is adjusted according to the relative spatial position.

[0026] Optionally, the navigation method further includes:

[0027] Get the screen recording command;

[0028] Recording the video after the navigation mark and the navigation reference object are anchored according to the screen recording instruction;

[0029] The video is sent to a preset terminal.

[0030] Optionally, the navigation method further includes:

[0031] When the navigation reference includes a road intersection and a marker, the augmented reality device issues a voice message, and the voice message is used to guide the user through the road intersection, and / or, when the navigation reference includes a road intersection and a marker, the navigation mark includes an arrow mark having the characteristics of the marker, and the arrow mark guides the user through the road intersection.

[0032] Optionally, the navigation method further includes:

[0033] Generating a first navigation route in the augmented reality device according to the current location of the augmented reality device and a preset destination;

[0034] Acquire a second navigation route, where the second navigation route is generated by the mobile terminal according to the current location and a preset destination;

[0035] When the first navigation route and the second navigation route are different, a reminder instruction is generated, where the reminder instruction is used to remind the user to detect the communication status of the mobile terminal.

[0036] The present application also discloses a navigation device, which comprises:

[0037] An image module, the image module is used to obtain an environment image of the augmented reality device within a predetermined range of the current position;

[0038] A recognition module, the recognition module is used to recognize the navigation reference object in the environment image;

[0039] An acquisition module, the acquisition module is used to acquire preset content of the navigation reference object;

[0040] A display module is used to convert the preset content into a navigation mark for display, wherein the navigation mark and the navigation reference are in an anchored state.

[0041] The present application also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a navigation program, and when the navigation program is executed by a processor, the navigation method of the augmented reality device as described above is implemented.

[0042] The present application also discloses a computer device, which includes a computer-readable storage medium, a processor, and a navigation program stored in the computer-readable storage medium. When the navigation program is executed by the processor, the navigation method of the augmented reality device as described above is implemented.

[0043] The present invention discloses a navigation method for an augmented reality device, a computer device, and a computer-readable storage medium, which have the following technical effects:

[0044] This method anchors the displayed navigation mark in the navigation reference object, uses the navigation mark to provide navigation guidance to the user, realizes the effective integration of the navigation mark and the real environment, and improves the AR navigation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1This is a flowchart of a navigation method for an augmented reality device according to Embodiment 1 of the present invention;

[0046] Figure 2 A flow chart of a method for determining a current position of an augmented reality device according to Embodiment 1 of the present invention;

[0047] Figure 3 This is a schematic diagram of determining the current position of an augmented reality device according to the first embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of anchoring a navigation marker and a navigation reference object according to the first embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of anchoring a navigation marker and a navigation reference object in a driving scenario according to the first embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of anchoring a navigation marker and a navigation reference object in a walking scene according to the first embodiment of the present invention;

[0051] Figure 7 is a principle block diagram of a navigation device according to a second embodiment of the present invention;

[0052] Figure 8 Schematic diagram of a computer device according to a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0054] Before describing the various embodiments of the present application in detail, the technical concept of the present application is first briefly described: when using AR glasses for navigation, the navigation page on a mobile terminal such as a mobile phone is usually directly displayed on the display interface of the AR glasses, which cannot be effectively integrated with the real environment, reducing the guidance and intuitiveness of the navigation. To this end, the navigation method of the augmented reality device provided by the present application identifies the navigation reference objects in the environmental image within a predetermined range of the augmented reality device, generates and displays a navigation mark based on the navigation reference objects, and uses the navigation mark to provide navigation guidance to the user. The navigation mark can be effectively integrated with the environment, which improves the guidance and intuitiveness of the navigation process. The navigation method of the augmented reality device of the present application is described below in conjunction with more embodiments.

[0055] Specifically, Figure 1 As shown, the navigation method of the augmented reality device of the first embodiment includes:

[0056] Step S10, obtaining an environment image of the augmented reality device within a predetermined range of the current position;

[0057] Step S20: identifying navigation reference objects in the environment image;

[0058] Step S30, obtaining preset content of the navigation reference object;

[0059] Step S40: converting the preset content into a navigation mark for display, wherein the navigation mark and the navigation reference object are in an anchored state.

[0060] The navigation method of the first embodiment can be applied to a variety of navigation scenarios, including walking scenarios, cycling scenarios, driving scenarios, etc. In various navigation scenarios, the user may set the departure and destination in advance, or the user may not set the destination in advance. For example, the user sets the departure and destination in advance before preparing to drive out; for example, the user walks on the street without setting a clear destination. For the above various navigation scenarios, the navigation method of the first embodiment can generate navigation signs, and the navigation signs can be effectively integrated with the environment of each navigation scenario to guide the user more intuitively.

[0061] In the embodiments of the present disclosure, an augmented reality device is a device that can realize an augmented reality function, such as AR glasses, AR helmets, etc., or a MR device is also an augmented reality device when realizing the augmented reality function.

[0062] In step S10, the predetermined range refers to a local range in the user's visual environment. Among them, the visual environment is usually the physical environment of the user's visual field in front of the user, that is, the environment that the user pays more attention to. For example, when the user is moving, the visual field is usually forward. The visual environment can also be the physical environment of the user's side visual field. For the side visual field, the user is not currently paying attention, but may pay attention to it at a later moment. In one embodiment, the visual environment can include both the physical environment of the user's visual field in front of the user and the physical environment of the user's side visual field. Limited by the field of view (FOV) of the augmented reality device, the display interface size range of the augmented reality device is limited. The navigation mark corresponding to the navigation reference object outside the range limited by the field of view cannot be displayed on the display interface. Therefore, the predetermined range of this embodiment one is set to not exceed the range limited by the field of view of the augmented reality device, so that the navigation mark can be fully displayed. For other embodiments, when the augmented reality device has a display interface expansion function, when the augmented reality device is rotated to different directions to display different pictures or contents, the display range is increased. At this time, the predetermined range can be set to exceed the range limited by the field of view of the augmented reality device, and the navigation mark corresponding to the navigation reference outside the range limited by the field of view can be displayed by rotating the augmented reality device. For example, the camera captures an image with two navigation references R1 and R2, and both have been identified. Only the navigation reference R1 is within the field of view, and the navigation reference R2 is outside the field of view. At this time, the display interface can only display the navigation mark corresponding to the navigation reference R1. If the rotation scheme is adopted, the device can be rotated at this time so that the navigation reference R2 is within the field of view, and the navigation mark corresponding to the navigation reference R2 is displayed. At this time, there is no need to identify the navigation reference R2 again, as the identification has been completed before.

[0063] For example, an augmented reality device is usually equipped with multiple cameras, which are usually used to capture images of the user's eyes and the environment in front of the user. Therefore, in one embodiment, the camera of the augmented reality device is used to capture the user's field of view environment image, and the environment image within a predetermined range is intercepted from the field of view environment image. Of course, in other embodiments, a camera separated from the augmented reality device can also be used to obtain the environment image within a predetermined range.

[0064] Furthermore, after acquiring the environment image, an image recognition algorithm can be used to identify the navigation reference from the environment image. For example, a line recognition algorithm based on deep learning can be used to detect the navigation reference in the environment image. Exemplarily, a classic target detection algorithm based on the YOLO neural network model can be used to detect the navigation reference in the environment image; a classic target detection algorithm based on the R-CNN neural network model can also be used to detect the navigation reference in the environment image.

[0065] Further, for navigation scenes, the navigation reference objects of the first embodiment of the present invention generally include three basic target objects, namely, roads, buildings and natural landscapes, and for different navigation scenes, the navigation reference objects in the environment image generally appear in combination, for example, the more common combination is roads and buildings, roads and natural landscapes, of course, in some scenes, such as driving in an open environment such as a desert or grassland, the navigation reference object may be only a road, without buildings and natural landscapes. The roads here may only include highways, or may include highways and forks, crossroads, bends, pedestrian bridges, etc. on the highways. The buildings here may include commercial buildings, residential buildings, scenic spot buildings, etc. The natural landscape may include rivers, lakes, peaks, etc. In another embodiment, the navigation reference objects may also include traffic signs, such as common bus station signs, subway entrance signs, speed limit signs, highway intersection signs, etc. The navigation reference objects listed here are only the more common target objects in each navigation scene, and the types and quantities of navigation reference objects are not limited to these, and more navigation reference objects may be set according to actual needs to adapt to different navigation scenes.

[0066] Furthermore, it is also necessary to determine the current position of the augmented reality device to determine the relative position between the augmented reality device and the navigation reference object, so as to further determine the position of the navigation mark. For example, when the navigation mark is an arrow mark, it is necessary to determine the direction of the arrow mark based on the relative position between the augmented reality device and the navigation reference object.

[0067] Specifically, augmented reality devices are usually equipped with satellite positioning chips, which can be GPS positioning chips, Beidou satellite positioning chips, etc. Therefore, in one embodiment, when a user wears and turns on an augmented reality device, the satellite positioning chip on the augmented reality device can be used to determine the current position of the augmented reality device. In an open environment, or when the positioning accuracy is not high, this method can be used to obtain the current position of the augmented reality device. However, for scenes with dense buildings and high-speed driving, when using a satellite positioning chip to obtain the current position of the augmented reality device, there may be defects such as large positioning errors and positioning delays.

[0068] To this end, in another implementation, the current position of the augmented reality device is determined by using the geographical location of the reference object and the relative distance between the augmented reality device and the reference object, which is conducive to improving the positioning accuracy. Figure 2 As shown, the steps of determining the current position of the augmented reality device include the following:

[0069] Step S501: Use an augmented reality device to capture an image within a preset range of the augmented reality device;

[0070] Step S502: identifying a reference object in an image within a preset range of the augmented reality device and obtaining a geographical location of the reference object;

[0071] Step S503: Calculate the relative distance between the augmented reality device and the reference object;

[0072] Step S504: determine the current position of the augmented reality device according to the relative distance and the geographical position of the reference object.

[0073] For example, in one implementation, the preset range in step S501 depends on the shooting range of the camera of the augmented reality device, for example, the camera of the augmented reality device is used to shoot the field of view image in front of the augmented reality device. In another implementation, the image of the preset range of the augmented reality device captured in step S501 may be the environment image obtained in step S10, that is, the environment image may also be used to locate the augmented reality device.

[0074] Furthermore, in step S502, after acquiring the image of the preset range of the augmented reality device, an image recognition algorithm can be used to identify the reference object from the image. For example, a line recognition algorithm based on deep learning can be used to detect the reference object in the image. Exemplarily, the classic target detection algorithm based on the YOLO neural network model can be used to detect the reference object in the image; the classic target detection algorithm based on the R-CNN neural network model can also be used to detect the reference object in the image. The reference object can be a road, a building, a natural landscape, etc. In another embodiment, when the environmental image acquired in step S10 is used as the image of the preset range of the augmented reality device captured in step S501, the navigation reference object identified in step S20 can be used as the reference object in step S502.

[0075] Furthermore, after obtaining the reference object in step S502, step S503 obtains the relative distance between the augmented reality device and the reference object through a photo distance recognition method. The photo distance recognition method utilizes the principle of similar triangles. It is necessary to know the actual size of the object and the focal length of the camera, and then calculate the distance based on the pixel width of the object on the image. For example, when the reference object is identified as a certain business building, the actual height H of the business building can be determined based on the existing map database. For example, after identifying the business building, the name, height and other related information of the business building can be matched in the map databases such as Amap and Baidu Map. At the same time, the image height h corresponding to the business building can also be known. The focal length f of the camera is known, and the distance d from the camera to the business building can be calculated as (H*f) / h. In other embodiments, common distance measurement methods such as stereo triangulation method and ToF distance measurement can also be used.

[0076] In order to determine the current position of the augmented reality device, at least two reference objects are required, such as Figure 3 As shown, it is assumed that the two reference objects are reference object A and reference object B. The calculated relative distances between the user and reference object A and reference object B are d1 and d2 respectively.

[0077] Furthermore, in step S502, after identifying the reference object, it is also necessary to determine the geographical location of the reference object. According to the above description, the reference objects are all common, fixed objects and the relevant information has been included in the map database. For example, when the reference object is a business building, the name and geographical location of the business building can be queried in the map database, such as Amap, Baidu Map, etc. Due to the large number of map databases, the amount of queries and calculations brought about by direct matching are huge, and it is necessary to further narrow the area of ​​the map, that is, to use the local map data covering the reference object to determine the geographical location of the reference object. The local map data here can be directly obtained by the augmented reality device through networking, or it can be obtained through the user's mobile phone, tablet computer and other mobile terminals. The mobile terminal and the augmented reality device are connected through Bluetooth and other communication methods to transmit the local map data to the augmented reality device.

[0078] For example, in one implementation, the local map data is map navigation data generated according to the departure and destination set by the user, and the map navigation data includes route information and reference object information. Since the reference objects within the preset range of the augmented reality device are on the user's navigation route, the map navigation data must cover the reference objects, and the amount of data is greatly reduced compared to the map database.

[0079] In another embodiment, the local map data is map data covering the geographic location of the augmented reality device, and the local map data includes route information and reference object information adjacent to the geographic location of the augmented reality device. In this embodiment, the satellite positioning chip on the augmented reality device can be used to determine the geographic location of the augmented reality device, and the local map data can be obtained from the map database based on the geographic location. Exemplarily, the local map data can be local range map data with a radius of 100 meters centered on the geographic location of the augmented reality device.

[0080] Since the local map data covers the area near the augmented reality device, including the geographical location, name and other information of the reference object, the identified reference object can be matched with the local map data to filter the information of the identified reference object. After the reference object is identified from the image in step S502, the geometric features (height, width, length, shape and other features) and appearance features (color, pattern, etc.) of the reference object can be obtained. At this time, the corresponding reference object is filtered out from the local map data according to the geometric features, appearance features, etc. of the reference object, so as to determine the geographical location, name and other information of the identified reference object. For some reference objects, such as business buildings, there is a corresponding building name on the outside of the business building. At this time, by identifying the image, the name of the business building can also be directly identified. At this time, the corresponding reference object can be determined from the local map data according to the name, so as to determine the geographical location of the business building.

[0081] For example, in one embodiment, Figure 3 As shown, in step S502, it is assumed that reference object A and reference object B are identified from the image of the preset range of the augmented reality device, and the calculated relative distances between the augmented reality device C and reference object A and reference object B are d1 and d2 respectively, and the geographical locations of reference object A and reference object B have been determined. At the same time, since the distance d between reference object A and reference object B is fixed and can be obtained from the preset map data, the current location of the augmented reality device C can be determined according to the above relative distances d1 and d2 and the geographical locations of reference object A and reference object B. In essence, the augmented reality device C, reference object A and reference object B form a triangle, the three sides (d1, d2, d) of the triangle are known, and the positions of the two vertices (A, B) of the triangle are determined, so the position of the third vertex (C) of the triangle can be determined. It should be noted that the current position of the augmented reality device determined by this method may also be located at C', C' and C are symmetrical about the line AB, but according to the field of view environment, reference objects A and B are always located in front of the augmented reality device, and the restriction condition can be met at point C, but not at point C', that is, reference objects A and B are located behind point C', so point C' can be excluded, so that the position of point C can be determined as the current position of the augmented reality device. In summary, the current position of the augmented reality device with higher accuracy can be calculated through steps S501, S502, S503, and S504.

[0082] Furthermore, the preset content obtained in step S30 is the associated information of the navigation reference, such as the name, function, structure, rules and other extended content of the navigation reference. For example, when the navigation reference is a business building, the preset content may include the name, height, floor and other information of the business building. When the navigation reference is a highway intersection, the preset content may include the name of the highway intersection, the destination forecast information of each intersection and the like. For example, Figure 4 As shown, when the user is in a walking scene, after identifying the streets, oceans, and mountains in the environment image, the name of the street, the name and height of the ocean, and the name and distance of the mountain can be obtained, and the corresponding navigation logo can be generated, and the navigation logo can be anchored to each navigation reference object, so that the user can learn more about each reference object. For another example, when driving on a highway, the navigation reference objects in the environment image are identified, including forks in the road, distant mountains, etc., and the name of the fork in the road, a brief introduction to the direction, and the name and altitude of the mountain can be obtained, and corresponding navigation logos can be generated respectively, and the navigation logos can be anchored to the forks in the road and the mountains accordingly.

[0083] In step S40, the preset content is converted into a navigation mark for display. Exemplarily, the navigation mark is displayed using an augmented reality method, and the navigation mark is used to provide navigation guidance to the user. According to the above description, the navigation mark can be applied to various navigation scenarios. The following describes the navigation scenario with a departure point and a destination set in advance and the navigation scenario without a destination set in advance, respectively. In each scenario, the user wears an augmented reality device. It can be understood that the navigation mark displayed by the augmented reality device is a navigation mark anchored to a navigation reference object.

[0084] For navigation scenarios where the departure and destination are set in advance, in this navigation scenario, the user has a clear purpose. At this time, the route information of the map navigation data and the augmented reality device can be combined at the current location to display a more instructive navigation sign to indicate the user's direction of travel. For example, in a driving scenario, when the user is driving on a highway and faces an intersection, that is, the navigation reference includes the highway and the intersection. If it is determined according to the map navigation data that the user should go straight, the displayed navigation sign is a straight arrow, and the straight arrow points forward from the user along the straight direction of the highway; if it is determined according to the map navigation data that the user should turn left and go straight, the displayed navigation sign is a broken line arrow, and the straight arrow points from the user along the highway to the left intersection to guide the user. In another example, for a walking scenario, when a user is walking on the sidewalk and faces a fork in the road, if it is determined based on the map navigation data that the user should continue going straight along the original sidewalk, the displayed navigation mark will be a straight arrow, which points forward from the user along the straight direction of the sidewalk; if it is determined based on the map navigation data that the user should go toward the fork in the road, the displayed navigation mark will be a curved arrow, which points along the road from the user to the left intersection, so as to guide the user.

[0085] For navigation scenarios where no destination is set in advance, in this navigation scenario, the user does not have a clear destination, such as a user walking on the sidewalk, but at this time, the corresponding navigation logo can be displayed in combination with the current position of the augmented reality device to prompt the user of the possible direction of travel. For example, for a driving scenario, when a user is driving on a highway and is about to reach an intersection, the navigation logo displayed at this time can be multiple arrows, each arrow pointing to a different intersection, to prompt the user that there are multiple directions to choose from next; or when a user is driving on a highway and is about to turn, the navigation logo displayed at this time can be a turning arrow to prompt the user that he is about to enter a curve. Similarly, for a walking scenario, when the user is walking, when it is recognized that there are multiple forks and pedestrian bridges in the field of vision, the navigation logo displayed at this time may include multiple arrows, each arrow pointing to each fork and pedestrian bridge, to prompt the user that there are multiple directions to choose from next.

[0086] It should be noted that the navigation mark in the above example is an arrow, but it is not limited thereto, and other marks that can play a guiding role can be used as navigation marks. The above navigation mark can be a two-dimensional mark or a three-dimensional mark.

[0087] Further, in step S40, the navigation mark and the navigation reference are in an anchored state, which improves the display accuracy of the navigation mark, so that the navigation mark is better integrated with the real environment, and the guidance and intuitiveness of navigation are improved. Among them, it is first necessary to determine the geometric features and spatial position of the navigation reference, that is, the geometric features include shape, size, etc. For example, when the navigation reference is a highway, the width of the highway (or the number of lanes of the highway), whether the highway is a straight section or a turning section, etc. can be identified, and the spatial position is the position of the navigation reference in the environment. Further, according to the geometric features and spatial position of the navigation reference, the shape, size and position of the navigation mark are adjusted so that the navigation mark and the reference are more effectively integrated. For example, when the navigation reference is a highway and an intersection, the generated navigation mark is a left turn arrow. At this time, the shape and size of the left turn arrow need to be adapted to the reference. Finally, according to the spatial position of the navigation reference, the left turn arrow is anchored on the reference, so as to achieve the effective integration of the navigation mark and the real environment and guide the user more intuitively.

[0088] Furthermore, the navigation method of the first embodiment also includes: detecting the relative spatial posture of the augmented reality device and the navigation reference object; and adjusting the display position and / or display size of the navigation mark according to the relative spatial posture.

[0089] Specifically, when a user wears an augmented reality device to walk or drive, the user's field of view is constantly moving. The continuous movement of the user's field of view can be equivalent to the continuous change of the current position of the augmented reality device and the continuous change of the relative spatial posture of the augmented reality device with respect to the navigation reference object. Therefore, after obtaining and displaying the corresponding navigation mark based on the environmental image within a predetermined range of the current position of the augmented reality device, further adjusting the display position and display size of the navigation mark according to the relative spatial posture can make the navigation mark more harmonious with the real environment.

[0090] In one embodiment, over a short period of time, the current position of the augmented reality device is constantly changing, and the navigation reference objects in the environmental image are the same. For example, while walking, the navigation reference objects in the user's field of view are the same buildings and intersections, and the type of navigation signs displayed at this time remains unchanged. During the continuous change of the current position of the augmented reality device, since the navigation reference object is stationary, the relative spatial position of the augmented reality device and the navigation reference object is constantly changing. Under the premise that the type of displayed navigation sign remains unchanged, the display position and size of the navigation sign need to be dynamically adjusted. In this embodiment, the relative spatial position of the augmented reality device with respect to the navigation reference object is mainly the relative position and relative distance between the augmented reality device and the navigation reference object. For example, in Figure 5The navigation signs in the image include an arrow sign pointing to the highway intersection. During the process of moving forward, the distance between the augmented reality device and the highway intersection gradually decreases. At this time, the length and width of the arrow sign are proportionally reduced to adapt to the change in distance. At the same time, the relative spatial position between the augmented reality device and the highway intersection also changes. At this time, the angle of the arrow sign needs to be dynamically adjusted to keep the arrow sign pointing to the highway intersection.

[0091] In another embodiment, within a short period of time, the current position of the augmented reality device remains unchanged, and the navigation reference in the environmental image is the same. For example, when the user stops at an intersection, the user turns his head to observe different intersections, that is, the augmented reality device rotates. Since the navigation reference is stationary, the relative spatial position of the augmented reality device and the navigation reference changes. In this embodiment, the relative spatial position of the augmented reality device with respect to the navigation reference is mainly the relative angle between the display interface of the augmented reality device and the navigation reference. Therefore, the display position and display size of the navigation mark need to be dynamically adjusted to adapt to the change in relative angle. In this embodiment, the rotation angle of the augmented reality device can be obtained through the sensor carried by the augmented reality device itself, such as an IMU sensor, to further adjust the display position and display size of the navigation mark.

[0092] Furthermore, the navigation method of the first embodiment of the present invention also includes: detecting the illumination intensity of the environment image; adjusting the display parameters when converting the preset content into the navigation logo for display according to the illumination intensity. The display parameters here include display brightness, opacity, etc., so that the display of the navigation logo can adapt to the lighting conditions in the environment. Among them, the illumination intensity can be determined by detecting the pixel grayscale value of the environment image, and the larger the pixel grayscale value, the greater the illumination intensity. Opacity indicates the degree of transparency of the displayed content in the display interface. For example, the smaller the opacity of the application icon on the display interface, the more transparent the application icon is, and the more difficult it is for the user to see clearly, and vice versa. The easier it is to see clearly. When it is detected that the illumination intensity is large, the display brightness and opacity of the navigation logo can be increased at this time to prevent the navigation logo from being displayed clearly in a strong light environment so that it can be seen by the user.

[0093] Furthermore, the navigation method of the first embodiment of the present invention further includes: obtaining a screen recording instruction; recording a video after the navigation mark and the navigation reference object are anchored according to the screen recording instruction; and sending the video to a preset terminal. Exemplarily, the preset terminal here can be a mobile terminal of a user wearing an augmented reality device, or a mobile terminal of another user, or a display screen on a car. For example, for a driving scene, when the driver wears an augmented reality device, after the augmented reality device displays the navigation mark and the navigation mark and the navigation reference object are anchored, the video after the navigation mark and the navigation reference object are anchored is recorded according to the screen recording instruction, and the video is sent to the display screen in the back row of the car and displayed on the display screen, so that the rear passengers can show the augmented reality navigation process and increase the fun. Among them, the screen recording instruction is generated according to the corresponding operation of the user, for example, the screen recording instruction can be generated by recognizing the user's voice through the augmented reality device, or the screen recording instruction can be generated by detecting the user's touch action through the touch area on the augmented reality device.

[0094] Furthermore, the navigation method of the first embodiment further includes: when the navigation reference includes a road intersection and a landmark, the augmented reality device issues a voice message, and the voice message is used to guide the user to pass through the road intersection. Exemplarily, when the user drives to the intersection, the displayed navigation mark is a left turn icon, instructing the user to turn left and then move forward. If the identified navigation reference includes a coffee shop located at the left intersection, a voice message "Please turn left and go along the street with the coffee shop" can be generated and played through the speaker or earphone of the augmented reality device.

[0095] In another embodiment, when the navigation reference includes a road intersection and a landmark, the navigation mark includes an arrow mark having the characteristics of the landmark, and the arrow mark guides the user to pass through the road intersection. When the user walks to the intersection, the displayed navigation mark is a left turn icon, instructing the user to turn left and move forward. If the identified navigation reference includes a coffee shop located at the left intersection, a pattern mark of the coffee shop can also be generated and superimposed on the left turn icon as an arrow mark to better guide the user to move forward.

[0096] Furthermore, the navigation method of the first embodiment of the present invention further includes: generating a first navigation route in the augmented reality device according to the current position and the preset destination of the augmented reality device; obtaining a second navigation route, the second navigation route is generated by the mobile terminal according to the current position and the preset destination; when the first navigation route and the second navigation route are different, generating a reminder instruction, the reminder instruction is used to remind the user to detect the communication status of the mobile terminal. When the user uses the augmented reality device worn and the mobile terminal carried for navigation, the navigation results of the two can be comprehensively verified. On the one hand, the user can be reminded to check the network signal, GPS signal, etc. of the mobile terminal according to the comprehensive verification result. On the other hand, after confirming that the communication status of the mobile terminal is good, the optimal route can be selected from the first navigation route and the second navigation route. Among them, when comparing and judging the first navigation route and the second navigation route, it can be compared whether the route shapes of the first navigation route and the second navigation route are completely overlapped. If they are overlapped, it means that the first navigation route and the second navigation route are the same, otherwise they are not the same. In addition, the generated reminder instruction can be a voice instruction or a vibration instruction. On the one hand, the augmented reality device can make corresponding voice or generate corresponding vibration according to the reminder instruction. On the other hand, the reminder instruction can be sent to the mobile terminal. The mobile terminal makes corresponding voice or generates corresponding vibration according to the reminder instruction to remind the user.

[0097] In order to increase the richness of navigation, more navigation signs of other display types may be displayed.

[0098] In one embodiment, after the navigation reference object is identified in step S20, the relative distance between the augmented reality device and the navigation reference object can be calculated using the photo distance recognition method, and the relative distance can be displayed on the display interface through a distance mark, so that the user can understand the distance from the navigation reference object. Figure 5 As shown, when driving on a highway, when a relatively far intersection is identified and the relative distance from the user to the intersection is calculated, a corresponding distance mark can be generated, the distance mark can be anchored to the intersection and displayed on the display interface of the augmented reality device, so as to inform the user of the distance from the intersection in advance and prepare for deceleration and braking in time. In another example, Figure 6 As shown, when walking on the street, when the identified navigation reference objects are multiple buildings, after calculating the relative distance from the user to each building, the corresponding distance markers can be generated, and each distance marker can be anchored to each building and displayed on the display interface of the augmented reality device.

[0099] Furthermore, for various navigation scenarios, the navigation references identified from the environmental image usually also include traffic signs. For various types of traffic signs, navigation signs with traffic signs can be displayed, and the navigation signs of traffic signs are anchored to each traffic sign and displayed on the display interface of the augmented reality device, and the navigation signs of traffic signs are used to remind users. For example, if the traffic signs are signs such as speed limit, camera, bus stop, school ahead, etc., the corresponding navigation signs can be displayed to remind users in advance, and to make it easier for users to notice and avoid missing roadside traffic signs during driving.

[0100] Furthermore, more navigation signs may be displayed in combination with specific navigation scenarios to assist the user.

[0101] For example, in one embodiment, for each navigation scenario, after detecting the current position of the augmented reality device, the current position can be displayed on the display interface of the augmented reality device, and the current position can be represented by longitude and latitude. Furthermore, a direction mark can be displayed on the display interface to indicate the current direction of travel of the augmented reality device.

[0102] In another embodiment, for a navigation scenario in which a departure point and a destination are pre-set, a destination sign and a remaining distance and a remaining time from the user's geographical location to the destination may be displayed on the display interface.

[0103] like Figure 7 As shown, the second embodiment also discloses a navigation device, which includes an image module 100, a recognition module 200, an acquisition module 300 and a display module 400. The image module 100 is used to acquire an environment image of the augmented reality device within a predetermined range of the current position, the recognition module 200 is used to identify a navigation reference in the environment image, the acquisition module 300 is used to acquire preset content of the navigation reference, and the display module 400 is used to convert the preset content into a navigation mark for display, wherein the navigation mark and the navigation reference are in an anchored state.

[0104] Furthermore, the image module 100 is also used to capture an image within a preset range of the augmented reality device, the recognition module 200 recognizes a reference object in the image within a preset range of the augmented reality device and obtains the geographical location of the reference object, and the acquisition module 300 is also used to calculate the relative distance between the augmented reality device and the reference object and to determine the current position of the augmented reality device based on the relative distance and the geographical location of the reference object.

[0105] Furthermore, the navigation device also includes a positioning module 500, which is used to obtain the geographical location of the augmented reality device according to a satellite positioning method. The acquisition module 300 is also used to obtain local map data covering the geographical location. The identification module 200 is also used to match a reference object in the local map data to obtain the geographical location of the reference object.

[0106] Furthermore, the image module 100 is further used to detect the light intensity of the environment image, and the display module 400 is further used to adjust the display parameters when the preset content is converted into the navigation mark for display according to the light intensity.

[0107] The navigation device also includes a posture module 600, which is used to detect the relative spatial posture of the augmented reality device and the navigation reference object, and the display module 400 is used to adjust the display position and / or display size of the navigation mark according to the relative spatial posture.

[0108] Furthermore, the navigation device further includes a voice module 700. When the navigation reference includes a road intersection and a landmark, the voice module 700 is used to issue a voice message, and the voice message is used to guide the user to pass through the road intersection.

[0109] Further, when the navigation reference includes a road intersection and a landmark, the navigation mark displayed by the display module 400 includes an arrow mark having the characteristics of the landmark, and the arrow mark guides the user to pass through the road intersection.

[0110] Furthermore, the navigation device also includes a navigation module 800, which is used to generate a first navigation route in the augmented reality device according to the current position and a preset destination of the augmented reality device. The acquisition module 300 is also used to acquire a second navigation route, which is generated by the mobile terminal according to the current position and the preset destination. When the first navigation route and the second navigation route are different, the navigation module 800 is also used to generate a reminder instruction, which is used to remind the user to check the communication status of the mobile terminal.

[0111] The third embodiment also discloses a computer-readable storage medium, which stores a navigation program. When the navigation program is executed by a processor, the navigation method of the augmented reality device in the first embodiment is implemented.

[0112] This fourth embodiment also discloses a computer device, at the hardware level, such as Figure 8As shown, the computer device includes a processor 12, an internal bus 13, a network interface 14, and a computer-readable storage medium 11. The processor 12 reads the corresponding computer program from the computer-readable storage medium and then runs it, forming a request processing device at the logical level. Of course, in addition to software implementations, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logical unit, but can also be hardware or logic devices. A navigation program is stored on the computer-readable storage medium 11, and when the navigation program is executed by the processor, the navigation method of the above-mentioned augmented reality device is implemented.

[0113] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0114] The computer device mentioned above may be an augmented reality device. For example, the computer device may be augmented reality or virtual reality glasses.

[0115] In the example of augmented reality or virtual reality glasses, the computer device can be configured to pass data to an external processing device and receive data from the external processing device through a signal connection, and the signal connection can be a wired connection, a wireless connection or a combination thereof. However, in other cases, the computer device can be used as a stand-alone device, that is, data processing is performed on the computer device itself. The signal connection can be configured to carry any kind of data, such as image data (e.g., still images and / or full motion video, including 2D and 3D images), audio, multimedia, voice and / or any other type of data. The external processing device can be, for example, a game console, a personal computer, a tablet computer, a smart phone or other types of processing devices. The signal connection can be, for example, a universal serial bus (USB) connection, a Wi-Fi connection, a Bluetooth or Bluetooth low energy (BLE) connection, an Ethernet connection, a cable connection, a DSL connection, a cellular connection (e.g., 3G, LTE / 4G or 5G), etc. or a combination thereof. Additionally, the external processing device can communicate with one or more other external processing devices via a network, and the network can be or include, for example, a local area network (LAN), a wide area network (WAN), an intranet, a metropolitan area network (MAN), the global Internet or a combination thereof.

[0116] The computer device may be equipped with a display component, an optical device, a sensor, a processor, etc. In the example of augmented reality or virtual reality glasses, the display component is designed to, for example, implement the functions of virtual reality glasses by projecting light into the user's eyes, for example, by projecting light into the user's eyes, overlaying images on the user's view of their real world environment to implement the functions of augmented reality glasses. The computer device may also include an ambient light sensor, and may also include an electronic circuit system to control at least some of the above components and perform associated data processing functions. The electronic circuit system may include, for example, one or more processors and one or more memories.

[0117] The specific implementation methods of the present invention are described in detail above. Although some embodiments have been shown and described, those skilled in the art should understand that these embodiments can be modified and improved without departing from the principles and spirit of the present invention, the scope of which is defined by the claims and their equivalents. These modifications and improvements should also be within the scope of protection of the present invention.

Claims

1. A navigation method for an augmented reality device, characterized in that: The navigation method comprises: Acquire an environment image of the augmented reality device within a predetermined range of the current position; identifying a navigation reference object in the environment image; Acquire preset content of the navigation reference object; The preset content is converted into a navigation mark for display, and the navigation mark and the navigation reference object are in an anchored state.

2. The navigation method of the augmented reality device according to claim 1, characterized in that: The navigation method further includes determining a current position of the augmented reality device, wherein determining the current position of the augmented reality device includes: Using the augmented reality device to capture an image within a preset range of the augmented reality device; Identifying a reference object in an image within a preset range of the augmented reality device and acquiring a geographic location of the reference object; Calculating the relative distance between the augmented reality device and the reference object; The current position of the augmented reality device is determined according to the relative distance and the geographical position of the reference object.

3. The navigation method of the augmented reality device according to claim 2, characterized in that: The acquiring the geographical location of the reference object comprises: Obtaining the geographic location of the augmented reality device according to a satellite positioning method; Acquiring local area map data covering the geographical location; The reference object is matched in the local map data to obtain the geographical location of the reference object.

4. The navigation method of the augmented reality device according to claim 1, characterized in that: The navigation method further comprises: Detecting the light intensity of the environment image; The display parameters when the preset content is converted into a navigation mark for display are adjusted according to the light intensity.

5. The navigation method of the augmented reality device according to claim 1, characterized in that: The navigation method further comprises: Detecting the relative spatial position of the augmented reality device and the navigation reference object; The display position and / or display size of the navigation mark is adjusted according to the relative spatial position.

6. The navigation method of the augmented reality device according to claim 1, characterized in that: The navigation method further comprises: Get the screen recording command; Recording the video after the navigation mark and the navigation reference object are anchored according to the screen recording instruction; The video is sent to a preset terminal.

7. The navigation method of the augmented reality device according to claim 1, characterized in that: The navigation method further comprises: When the navigation reference includes a road intersection and a marker, the augmented reality device issues a voice message, and the voice message is used to guide the user through the road intersection, and / or, when the navigation reference includes a road intersection and a marker, the navigation mark includes an arrow mark having the characteristics of the marker, and the arrow mark guides the user through the road intersection.

8. The navigation method of the augmented reality device according to claim 1, characterized in that: The navigation method further comprises: Generating a first navigation route in the augmented reality device according to the current location of the augmented reality device and a preset destination; Acquire a second navigation route, where the second navigation route is generated by the mobile terminal according to the current location and a preset destination; When the first navigation route and the second navigation route are different, a reminder instruction is generated, where the reminder instruction is used to remind the user to detect the communication status of the mobile terminal.

9. A navigation device, characterized in that: The navigation device comprises: An image module, the image module is used to obtain an environment image of the augmented reality device within a predetermined range of the current position; A recognition module, the recognition module is used to recognize the navigation reference object in the environment image; An acquisition module, the acquisition module is used to acquire preset content of the navigation reference object; A display module is used to convert the preset content into a navigation mark for display, wherein the navigation mark and the navigation reference are in an anchored state.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a navigation program, and when the navigation program is executed by a processor, the navigation method of the augmented reality device according to any one of claims 1 to 8 is implemented.

11. A computer device, characterized in that: The computer device includes a computer-readable storage medium, a processor, and a navigation program stored in the computer-readable storage medium. When the navigation program is executed by the processor, the navigation method of the augmented reality device according to any one of claims 1 to 8 is implemented.

Images