Target guiding method based on AR guide board

Abstract

The invention discloses a target guiding method based on an AR (Augmented Reality) guide board. The method comprises the following steps: a, collecting real-time scene and position information and behavior data of a user in an AR environment through mobile phone equipment and an application program; b, identifying a road object and a place object in the current scene based on the real-time scene information of the user; c, performing name identification on the place object based on preset map information; d, performing direction identification on the road object based on preset map information, destination information of the user and user behavior data; and e, displaying the AR scene image with the name identifier and the direction identifier overlaid on the guide board in the real world to the user. The method can be applied to various outdoor places needing navigation guidance and can also be applied to indoor places needing navigation guidance, and the problems that existing indoor place navigation is low in positioning precision, poor in user experience and the like are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of AR, and particularly relates to a target guiding method based on an AR guide sign. Background Art

[0002] Currently, indoor navigation is mainly implemented by traditional positioning technologies based on wireless signals, navigation technologies based on two-dimensional codes / markers, or traditional 2D flat navigation interfaces. Among them, traditional positioning technologies based on wireless signals have defects such as insufficient positioning accuracy and high costs; navigation technologies based on two-dimensional codes / markers have defects such as cumbersome operations and lack of dynamic path updates, while traditional 2D flat navigation interfaces are not user-friendly to people with poor sense of direction and lack scene information, and cannot intuitively display the three-dimensional positional relationships of key landmarks.

[0003] Three-dimensional holographic codes are immersive three-dimensional interactive simulation systems. They utilize AR augmented reality technology to superimpose digital information on the user's real vision, enabling users to interact and operate with this virtual information in a virtual environment, providing users with a richer and more immersive experience. Therefore, if AR augmented reality technology can be applied to indoor navigation to solve problems such as low positioning accuracy, poor user experience, and insufficient environmental adaptability existing in existing indoor navigation technologies, it has become a research hotspot. Summary of the Invention

[0004] The present invention provides a target guiding method based on an AR guide sign to solve problems such as low positioning accuracy, poor user experience, and insufficient environmental adaptability existing in traditional indoor navigation technologies.

[0005] To achieve the purpose of the present invention, the present invention provides a target guiding method based on an AR guide sign, and the method includes the following steps:

[0006] a. Collect real-time scene, location information, and behavior data of the user in the AR environment through a mobile device and an application program;

[0007] b. Identify road objects and place objects in the current scene based on the user's real-time scene information;

[0008] c. Identify the name of the place object based on preset map information;

[0009] d. Identify the direction of the road object based on preset map information, the user's destination information, and the user's behavior data;

[0010] e. Display an AR scene image with a name identifier and a direction identifier on a guide sign superimposed on the real world to the user.

[0011] In step a, the method for collecting real-time scene information of the user in the AR environment is:

[0012] Through the mobile phone camera module, take pictures at a specific frequency, and combine with the inertial measurement unit, gyroscope and accelerometer to obtain the user's real-time perspective scene and angle changes.

[0013] In step a, the method for collecting the user's real-time position information in the AR environment is as follows:

[0014] Obtain the user's real-time geographical location information by combining the Beidou positioning system and the WIFI positioning technology. The Beidou positioning system provides accurate positioning of the user outdoors, and the WIFI positioning technology provides accurate positioning of the user in the indoor environment.

[0015] In step a, the method for collecting the user's behavior data is as follows:

[0016] By calling the platform AR environment API, obtain the user's interaction actions with virtual objects in real time, including operations such as touch, click, and long press, as well as the user's stay time in a specific scene.

[0017] Further, the platform AR environment API includes the interface provided by ARKit in the ISO environment or the interface provided by ARCore in the Android environment.

[0018] In step d, if the user does not set the destination information, at least one road object is directionally marked based on the user's current orientation to facilitate the guidance of the place object in the current scene.

[0019] In step e, the AR scene image further includes a destination identifier, and the destination identifier is set beside the direction identifier, and the extension direction of the destination identifier is the same as the indication direction of the direction identifier.

[0020] In some embodiments, in step e, the AR scene image further includes key place objects, the key place objects are highlighted, and a guiding direction is displayed on the key place objects.

[0021] Further, the key place objects are calculated based on the destination position and the position of the place object at the current position.

[0022] In some embodiments, in step e, if the identification area of the adjacent direction identifier shown in the current screen is less than the set value, the place object that blocks the direction identifier is made transparent so that the adjacent direction identifier shown in the current screen is equal to the set value.

[0023] The beneficial effects of the present invention are as follows: The target guiding method based on the AR guiding sign in this embodiment of the present invention obtains road objects and place objects in the current scene according to the real-time scene information in the AR environment, names the place objects, and marks the directions of the road objects, forming an AR scene image with name and direction marks and superimposing it on the guiding sign in the real world, which can provide users with more accurate, intuitive, vivid and convenient navigation guiding services. This method can be applied to various outdoor places that require navigation guiding, and can also be used for indoor places that require navigation guiding, solving the problems of low positioning accuracy and poor user experience existing in the existing indoor place navigation.

[0024] On the other hand, by setting destination marks and guiding key place objects, a convenient navigation guiding service is further provided for users. In addition, by making the place objects that block adjacent direction marks transparent, the interference caused by the place objects to the direction marks is avoided, ensuring that users can clearly and accurately identify the guidance of the guiding sign. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is the flowchart of the method of the present invention.

[0026] Figure 2 is a schematic diagram of the name mark of the place object in an embodiment of the present invention.

[0027] Figure 3 is a schematic diagram of the AR scene image in another embodiment of the present invention.

[0028] Figure 4 is a schematic diagram of the AR scene image when no destination information is set in the present invention.

[0029] Figure 5 is a schematic diagram of the AR scene image when the place object blocking the direction mark is made transparent in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present invention.

[0031] Refer to Figure 1 , the target guiding method based on the AR guiding sign of the present invention superimposes virtual navigation information on the guiding sign in the real world to achieve real-time interaction with the guiding sign. The present invention can be applied to various places that require navigation guiding, such as shopping malls, scenic spots, hospitals, transportation hubs, etc.

[0032] Such asFigure 1 As shown in Figure 1 , the target guiding method based on the AR guiding sign of the present invention includes the following steps:

[0033] S10. Collect the real-time scene, location information and behavior data of the user in the AR environment through the mobile device and the application program.

[0034] In this step, in the AR-based indoor navigation system, the real-time collection and fusion of the user's real-time location information and behavior data are the core basis for realizing accurate navigation and immersive interaction. In this embodiment, the real-time perspective scene, location information of the user in the AR environment, and the interaction actions between the user and the virtual object are collected. Among them, the method for collecting the real-time scene of the user in the AR environment is: through the mobile phone camera module, shooting at a specific frequency, combined with the inertial measurement unit, gyroscope and accelerometer, to obtain the user's real-time perspective scene and angle changes. Specifically, a high-frame-rate camera (≥60fps) can be used to continuously capture environmental images, and a mobile phone supporting the HDR mode can be selected for shooting to cope with scenes with sudden light changes (such as indoor-outdoor transition areas). The inertial measurement unit synchronously collects the device angular velocity, linear acceleration and attitude angle data at a certain frequency to accurately obtain the user's real-time perspective scene and angle changes. The method for collecting the real-time location information of the user in the AR environment is: obtain the user's real-time geographical location information through the combination of the Beidou positioning system and the WIFI positioning technology. Among them, the Beidou positioning system provides accurate positioning of the user outdoors, and the WIFI positioning technology provides accurate positioning of the user in the indoor environment, ensuring that the user's location can be accurately obtained regardless of where the user is. The method for collecting the interaction actions between the user and the virtual object is: by calling the platform AR environment API, the interaction actions between the user and the virtual object are obtained in real time. Among them, the interaction actions between the user and the virtual object include operations such as touch, click, long press, etc., and the stay time of the user in a specific scene. In this embodiment, the AR environment APIs of different platforms are different. For example, the AR environment API in the ISO environment is the interface provided by ARKit, and the AR environment API in the Android environment is the interface provided by ARCore.

[0035] S20. Identify the road objects and place objects in the current scene based on the user's real-time scene information.

[0036] In this step, the system obtains the real-time scene, location information and behavior data of the user in the AR environment collected by the user's mobile device and the application program, and based on the user's current perspective scene, identifies the road objects and place objects in the current perspective scene. Among them, the road objects can be corridors, stairs, elevators, barrier-free passages, etc., and the place objects can be clinic doorplates, store LOGOs, fire-fighting facilities, etc.

[0037] S30. Identify the names of the place objects based on the preset map information.

[0038] In this step, the system pre-imports the map information of the current venue, and dynamically matches the recognized venue objects with the venues in the preset map to accurately identify the names of the venue objects. For example, Figure 2 in the illustrated embodiment, the venue object where the user is located is a hospital, and the name identifications of the venue object can be training rooms, offices, front desks, various departments, restrooms, etc.

[0039] S40. Identify the direction of the road object based on the preset map information, the user's destination information, and the user behavior data.

[0040] In this step, the system generates a dynamic adaptive AR direction guidance identification based on the preset map, the user-set destination information, and the user behavior data to identify the direction of the road object. Among them, this direction points to the direction of the destination, so as to facilitate the user to quickly find the destination. Specifically, as Figure 3 shown, this road object is an outdoor passage, and the user sets the destination as the meeting room. The system uses the user's current location and orientation as the starting direction and the destination direction as the ending direction to generate an indication arrow from the starting direction to the ending direction, thereby forming the corresponding direction identification. From Figure 3 it can be seen that the user's orientation is consistent with the destination ending direction. Therefore, it generates an indication arrow pointing forward, enabling the user to easily find the destination. When the user's orientation changes, the system automatically generates an indication arrow pointing to the destination direction according to the user's orientation, so that the user can accurately find the destination.

[0041] In some embodiments, as Figure 4 shown, if the user does not set the destination information, the direction of at least one road object is identified based on the user's current orientation, that is, to tell the user how many roads there are to go in the current scene, so as to facilitate the guidance of the venue objects in the current scene. Specifically, as Figure 4 shown in the illustrated embodiment, at this time the user does not set the destination information and the system does not collect the destination information. Then the system generates at least one road object for direction identification according to the collected real-time scene and angle of the user and the preset map information. For example, the user is currently in an aisle of a certain shopping mall, and there are multiple venue objects in front of the user's orientation, namely public restrooms, restaurants, cinemas, escalators, and vertical elevators. Then the system generates the direction identifications of the above five road objects for the user to select. If the user's orientation changes, the system re-identifies and generates the direction identifications of the multiple venue objects in the current orientation according to the user's orientation, so as to facilitate the guidance of each venue object in the current scene.

[0042] S50. Display an AR scene image with a name identifier and a direction identifier on the wayfinding sign superimposed on the real world to the user.

[0043] In this step, when the system generates the name identifier of the place object and the direction identifier of the road object in the user's current scene, it superimposes the AR scene image containing the name identifier and the direction identifier onto the real scene and displays it to the user, so as to provide the user with a more intuitive, vivid and convenient navigation guidance service.

[0044] In some embodiments, as Figure 3 shown, the AR scene image displayed by the system to the user further includes a destination identifier, which is set beside the direction identifier, and the extension direction of the destination identifier is the same as the indication direction of the direction identifier, so as to display the current guided destination to the user. Specifically, as Figure 3 shown in the embodiment, on the left side of the direction identifier, "Conference Room J2-1-1816" is marked. Among them, "Conference Room J2-1-1816" is the destination identifier, and the extension direction of the destination identifier is the same as the indication direction of the direction identifier. Of course, the destination identifier can also be marked on the right side of the direction identifier or other places convenient for the user to view.

[0045] In other embodiments, as Figure 3 shown, the AR scene image displayed by the system to the user further includes a key place object, which can more conveniently provide the user with the direction guidance to the destination. Among them, the key place object is calculated based on the destination position and the position of the place object at the current position. The key place object can be a turning point to the destination position, an entrance of a barrier-free elevator, etc. In addition, the system will generate different key place objects according to the different positions of the user. For example, the current key place object is a turning point, and the next key place object may be an elevator entrance, etc. In order to facilitate the user to accurately identify the key place object, the system highlights the key place object so that the user can clearly know the specific route to the destination. Specifically, as Figure 3 shown in the embodiment, the user's destination is a conference room and he needs to enter through the J2 entrance. At this time, the J2 entrance is the key place object, and a sign indicating the J2 entrance is displayed at this entrance, informing the user that he needs to enter through the J2 entrance to reach the conference room. In addition, the guiding direction can also be added to the sign of the key place object, that is, how the user needs to walk after reaching this key place, which is convenient for the user to perform the next operation after reaching the key place.

[0046] In other embodiments, as Figure 5As shown, if the identification area of the adjacent direction identification shown on the current screen is smaller than the set value, the system will perform transparency processing on the venue object that obscures the direction identification, so that the adjacent direction identification shown on the current screen is equal to the set value, thus not affecting the user experience. Among them, the adjacent direction identification is the direction identification located near a certain venue object. At this time, the direction identification is obscured by the venue object. Specifically, as Figure 5 shown in the embodiment, when the direction identification is located between two venue objects and the user walks to the position adjacent to the direction identification, at this time, part of the direction identification will be obscured by the venue object located in the lower right corner, resulting in the identification area of the direction identification seen by the user being smaller than the set value, and even making it impossible for the user to find the correct direction identification. In order to enable the user to find the direction identification in a timely and accurate manner, here the system performs transparency processing on the venue object that obscures the direction identification, so that the identification area of the adjacent direction identification is equal to the set value, that is, to Figure 5 perform transparency processing on the venue object located in the lower right corner in, so as to ensure that the identification area of the adjacent direction identification is equal to the set value, in order to improve the user experience.

[0047] To sum up, the target guidance method based on the AR guide sign in this embodiment obtains the road objects and venue objects in the current scene according to the real-time scene information in the AR environment, performs name identification on the venue objects, and performs direction identification on the road objects, and forms an AR scene image with name identification and direction identification and superimposes it on the guide sign in the real world, which can provide users with more accurate, intuitive, vivid and convenient navigation guidance services. This method can be applied to various outdoor venues that require navigation guidance, and can also be used in indoor venues that require navigation guidance, solving the problems of low positioning accuracy and poor user experience existing in the existing indoor venue navigation.

[0048] On the other hand, by setting the destination identification and guiding the key venue objects, it further provides users with a convenient navigation guidance service. In addition, by performing transparency processing on the venue object that obscures the adjacent direction identification, it avoids the interference caused by the venue object to the direction identification, and ensures that the user can clearly and accurately identify the guidance of the guide sign.

[0049] Although the present invention has been disclosed through the above embodiments, the protection scope of the present invention is not limited thereto. Without departing from the concept of the present invention, the deformations, substitutions, etc. made to the above components will all fall within the scope of the claims of the present invention.

Claims

1. A target guiding method based on an AR guiding sign, characterized in that, The method includes the following steps: a. Collect the real-time scene, location information, and behavior data of the user in the AR environment through a mobile device and an application; b. Identify the road objects and place objects in the current scene based on the user's real-time scene information; c. Identify the names of the place objects based on the preset map information; d. Identify the directions of the road objects based on the preset map information, the user's destination information, and the user's behavior data; e. Display to the user an AR scene image with name identifications and direction identifications on a guide sign superimposed on the real world.

2. The target guiding method based on the AR guiding sign according to claim 1, wherein, In step a, the method for collecting the real-time scene information of the user in the AR environment is: Shoot at a specific frequency through the mobile phone camera module, and combine with an inertial measurement unit, a gyroscope, and an accelerometer to obtain the user's real-time perspective scene and angle changes.

3. The target guiding method based on the AR guiding sign according to claim 1, wherein, In step a, the method for collecting the real-time location information of the user in the AR environment is: Obtain the user's real-time geographical location information through a combination of the Beidou positioning system and WIFI positioning technology. The Beidou positioning system provides accurate positioning for the user outdoors, and the WIFI positioning technology provides accurate positioning for the user in an indoor environment.

4. The target guiding method based on the AR guiding sign according to claim 1, wherein, In step a, the method for collecting the user's behavior data is: Call the platform AR environment API to obtain the user's interaction actions with virtual objects in real time, including operations such as touch, click, and long press, as well as the user's stay time in a specific scene.

5. The target guiding method based on the AR guiding sign according to claim 4, characterized in that, The platform AR environment API includes the interfaces provided by ARKit in the ISO environment or the interfaces provided by ARCore in the Android environment.

6. The target guiding method based on an AR guiding sign according to claim 1, characterized in that In step d, if the user does not set destination information, identify the directions of at least one road object based on the user's current orientation to facilitate guiding the place objects in the current scene.

7. The target guiding method based on the AR guiding sign according to claim 1, wherein, In step e, the AR scene image further includes a destination identifier, which is set next to the direction identifier, and the extension direction of the destination identifier is the same as the indication direction of the direction identifier.

8. The target guiding method based on the AR guiding sign according to claim 1, wherein In step e, the AR scene image further includes key place objects, which are highlighted, and guiding directions are displayed on the key place objects.

9. The target guiding method based on the AR guiding sign according to claim 8, wherein, The key place objects are calculated based on the destination location and the location of the place objects at the current position.

10. The target guidance method based on the AR guide sign according to claim 1, wherein In step e, if the identification area of the adjacent direction identifier displayed in the current screen is less than the set value, make the place object that blocks the direction identifier transparent so that the adjacent direction identifier displayed in the current screen is equal to the set value.

Images