Display method and device, electronic equipment and storage medium

Abstract

The invention relates to a display method which is displayed by a display device, and the method comprises the following steps: generating a target path according to initial position information and destination position information; a processing module of the display equipment generates a three-dimensional image for navigation according to the target path; and projecting the three-dimensional image to the eyes of a target object.

Description

Technical Field

[0001] The present disclosure relates to the field of navigation technology, and in particular to a display method and device, an electronic device and a storage medium. Background Art

[0002] In recent years, the market for head-mounted display devices has developed rapidly and has broad market prospects. With the continuous advancement of technology and the continuous expansion of application scenarios, head-mounted devices have been widely used in multiple industries, including games, medical care, education, industry, and military. These devices project digital information into the wearer's field of view through the augmented reality technology of Mixed Reality (MR), and provide users with a more immersive and interactive experience by combining digital information with the real world and transmitting virtual objects, images or information in the real world, creating a new feeling.

[0003] At present, the environment scenes of mixed reality in most applications are relatively fixed, but when it is applied to the field of navigation, its environment scenes will change in real time, such as tourism navigation, indoor navigation, car navigation, etc. The existing mixed reality path navigation first collects images through multiple external cameras and combines the video of the mobile camera on the head-mounted display device to calculate the location of the scene, and then transmits the collected images and videos to the terminal device, which processes the video image architecture and then transmits the image to the display device; this method is highly dependent on external cameras and has a high latency in image transmission. Summary of the invention

[0004] The present disclosure provides a display method and device, an electronic device and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a display method is provided, which is displayed by a display device, and includes:

[0006] Generate a target path according to the starting location information and the destination location information;

[0007] The processing module of the display device generates a three-dimensional image for navigation according to the target path;

[0008] The three-dimensional image is projected into the eyes of the target object.

[0009] Based on the above solution, the method further includes:

[0010] Determining the starting position information;

[0011] The determining the starting position information includes:

[0012] Identify the scene image captured by the display device to obtain the starting position information;

[0013] or,

[0014] The starting position information is determined based on a positioning function.

[0015] Based on the above solution, before the display device collects the scene image, the method further includes:

[0016] The scene image is calibrated until the scene image includes the ground of the scene where the human eye is currently located.

[0017] Based on the above solution, identifying the scene image captured by the display device to obtain the starting position information includes:

[0018] Identifying feature points in the scene image;

[0019] The starting position information is determined according to the feature points.

[0020] Based on the above solution, determining the starting position information according to the feature point includes:

[0021] Determining relative position information between the display device and a reference object indicated by the feature point;

[0022] Determining first position information of the feature point in a preset map;

[0023] The starting position information of the starting position in the preset map is determined according to the relative position information and the first position information.

[0024] Based on the above solution, determining the relative position information between the display device and the reference object indicated by the feature point includes:

[0025] Determine horizontal position information and vertical position information of the display device in the scene;

[0026] Establishing a spatial coordinate system according to the horizontal position information and the vertical position information;

[0027] respectively determining first coordinates and second coordinates of the display device and the indicated reference object in the spatial coordinate system;

[0028] Relative position information between the display device and a reference object indicated by the feature point is determined according to the first coordinate and the second coordinate.

[0029] Based on the above solution, generating a target path according to the starting location information and the destination location information includes:

[0030] The target path is generated in the preset map according to the starting position information, the destination position information and a path selection strategy.

[0031] Based on the above solution, the path selection strategy includes at least one of the following:

[0032] The total length of the target path;

[0033] The congestion condition of the target path.

[0034] According to a second aspect of an embodiment of the present disclosure, a display device is provided, including:

[0035] A generation module, used for generating a target path according to the starting position information and the destination position information;

[0036] A processing module, used for generating a three-dimensional image for navigation according to the target path;

[0037] The projection module is used to project the three-dimensional image to the eyes of the target object.

[0038] Based on the above solution, the device also includes:

[0039] A determination module, used to determine the starting position information;

[0040] The determining module is specifically used for:

[0041] Identify the scene image captured by the display device to obtain the starting position information;

[0042] or,

[0043] The starting position information is determined based on a positioning function.

[0044] Based on the above solution, the device also includes:

[0045] The calibration module is used to calibrate the scene image until the scene image includes the ground of the scene currently located by the human eye.

[0046] Based on the above solution, the determination module is further used to:

[0047] Identifying feature points in the scene image;

[0048] The starting position information is determined according to the feature points.

[0049] Based on the above solution, the determination module is further used to:

[0050] Determining relative position information between the display device and a reference object indicated by the feature point;

[0051] Determining first position information of the feature point in a preset map;

[0052] The starting position information of the starting position in the preset map is determined according to the relative position information and the first position information.

[0053] Based on the above solution, the determination module is further used to:

[0054] Determine horizontal position information and vertical position information of the display device in the scene;

[0055] Establishing a spatial coordinate system according to the horizontal position information and the vertical position information;

[0056] respectively determining first coordinates and second coordinates of the display device and the indicated reference object in the spatial coordinate system;

[0057] Relative position information between the display device and a reference object indicated by the feature point is determined according to the first coordinate and the second coordinate.

[0058] Based on the above solution, the generation module is specifically used for:

[0059] The target path is generated in the preset map according to the starting position information, the destination position information and a path selection strategy.

[0060] Based on the above solution, the path selection strategy includes at least one of the following:

[0061] The total length of the target path;

[0062] The congestion condition of the target path.

[0063] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0064] a memory for storing processor-executable instructions;

[0065] A processor connected to the memory;

[0066] Wherein, the processor is configured to execute the method displayed above.

[0067] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a computer, the computer is enabled to execute the method displayed above.

[0068] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0069] It can be seen from the above embodiments that the technical solution provided by the present disclosure, after generating the target path according to the starting position information and the destination position information, the processing module of the display device generates a three-dimensional image for navigation according to the target path, and projects the three-dimensional image to the eyes of the target object. In this way, the processing module is set in the display device, and there is no need to use an external camera to collect images. The dependence on external cameras is small, and its scope of use is wider while reducing the cost of use. In addition, the processing module in the display device directly processes data without the need for data transmission with external devices, so that data processing can be achieved more quickly, and three-dimensional image navigation can be displayed to the target object in real time.

[0070] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0072] Figure 1 is a flowchart of a display method according to an exemplary embodiment;

[0073] Figure 2 is a flowchart of another display method according to an exemplary embodiment;

[0074] Figure 3 is a flowchart of another display method according to an exemplary embodiment;

[0075] Figure 4 is a flowchart of another display method according to an exemplary embodiment;

[0076] Figure 5 is a flowchart of another display method according to an exemplary embodiment;

[0077] Figure 6 is a flowchart of another display method according to an exemplary embodiment;

[0078] Figure 7 is a flowchart of another display method according to an exemplary embodiment;

[0079] Figure 8 is a structural block diagram of a display device according to an exemplary embodiment;

[0080] Fig. 9 The present invention is a structural block diagram of a display device according to an exemplary embodiment. DETAILED DESCRIPTION

[0081] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0082] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present invention. The singular forms of "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0083] It should be understood that although the terms first, second, third, etc. may be used to describe various information in embodiments of the present invention, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0084] In recent years, the combination of smart display devices and mixed reality technology has been applied in various industries. In the gaming industry, players can be fully immersed in the game world and enjoy a more realistic gaming experience; in the medical industry, it is used to simulate surgery and treatment processes to improve the skills and knowledge level of medical staff; in the education industry, it is used in virtual laboratories, simulated scenarios, course interactions, etc., so that students can better understand and master instructions; in industrial manufacturing, these devices can help workers complete tasks more efficiently; in the military field, they can provide real-time intelligence and tactical information to help soldiers better perform their tasks.

[0085] The display device provided in the embodiment of the present invention is applied to tourist navigation, which can provide tourists with an intuitive navigation experience during travel, helping them to better understand scenic spots and the surrounding environment; applied to indoor navigation, it can provide users with accurate navigation in indoor places such as large shopping malls, airports, hospitals, offices, etc.; applied to car navigation, it can directly transmit route instructions and real-time traffic information to the driver's field of view, improving the driver's safety and driving experience

[0086] In general, navigation technology that combines mixed reality technology with display devices has broad application prospects and can provide users with more intelligent and intuitive navigation services in multiple fields.

[0087] like Figure 1 As shown, an embodiment of the present disclosure provides a display method, which is displayed by a display device, including:

[0088] 101: Generate a target path according to the starting position information and the destination position information;

[0089] 102: generating, by a processing module of the display device, a three-dimensional image for navigation according to the target path;

[0090] 103: Projecting the three-dimensional image to the eyes of the target object.

[0091] In the embodiments of the present disclosure, the display device may be used in augmented reality (AR), virtual reality (VR), or mixed reality (MR). By utilizing VR, AR, or MR technology, a user using the display device can visually cause the real world and computer graphics, i.e., virtual information, to overlap, and after the overlap, the user can fully see the real world surrounding the user.

[0092] In some embodiments, the display device may be referred to as a smart display device. For example, the display device may be a pair of smart glasses or a head-mounted display device.

[0093] The head mounted display device may include but is not limited to smart glasses and / or a smart helmet.

[0094] In step 103, the three-dimensional image at least includes a navigation direction.

[0095] In the disclosed embodiment, when the three-dimensional image is projected onto the eyes of the target object, only the navigation direction is displayed to indicate that the target object has arrived at the destination, so that the navigation process is more concise and clear without affecting the target object's driving vision.

[0096] In the embodiment of the present disclosure, after the target path is generated according to the starting position information and the destination position information, the processing module of the display device generates a three-dimensional image for navigation according to the target path, and projects the three-dimensional image to the eyes of the target object. In this way, data is directly processed by the processing module in the display device without the need for data transmission with external devices, so data processing can be achieved more quickly, and three-dimensional image navigation can be displayed to the target object in real time.

[0097] In some embodiments, before step 101, the method further includes:

[0098] 100: Determine the starting position information;

[0099] Specifically, the starting location information indicates the location information of the user when starting navigation.

[0100] In some embodiments, Figure 2 As shown, in the above step 100, the determining of the starting position information includes:

[0101] 201: Identify the scene image captured by the display device to obtain the starting position information;

[0102] or,

[0103] 202: Determine the starting position information based on a positioning function.

[0104] In the embodiment of the present disclosure, in the above step 201, the display device captures a scene image through its own camera. It should be noted that the scene image here is a depth image.

[0105] A depth image, also known as a distance image, refers to an image that uses the distance between the image collector and each point in the captured scene as the pixel value.

[0106] The scene image captured by the display device is identified through a simultaneous localization and mapping (SLAM) algorithm to obtain the starting position information. In this way, the display device does not need to rely on an external camera to capture images, has little reliance on external cameras, and has a wider range of uses while reducing the cost of use.

[0107] In another embodiment, in the above step 202, the display device determines the location information of the user's location when the user starts navigation through a positioning function, and determines the location information as the starting location information.

[0108] Exemplarily, the positioning function may be implemented based on satellite positioning;

[0109] Exemplarily, the positioning function may also be implemented by base station assisted positioning;

[0110] Exemplarily, the positioning function may also be implemented based on positioning technologies such as Wireless Fidelity (Wi-Fi) positioning and Ultra Wide Band (UWB) positioning.

[0111] In the above step 201, before the display device acquires the scene image, the method further includes:

[0112] 200: Calibrate the scene image until the scene image includes the ground of the scene where the human eye is currently located.

[0113] In this way, the distance between the display device and the ground can be determined based on the ground in the scene image and in combination with the acquisition parameters when the display device acquires the scene image, so as to facilitate the subsequent determination of the position information of the display device.

[0114] Here, the acquisition parameters include but are not limited to the angle, focal length, etc. when the display device acquires the scene image.

[0115] In some embodiments, Figure 3 As shown, in the above step 201, the identifying the scene image captured by the display device to obtain the starting position information includes:

[0116] 2011: identifying feature points in the scene image;

[0117] 2012: Determine the starting position information according to the feature points.

[0118] In some embodiments, the feature points indicate reference objects in the real scene corresponding to the scene image.

[0119] In the above step 2011, the optical character recognition (OCR) technology is used to identify the feature point identifiers corresponding to the feature points in the scene image.

[0120] In some embodiments, the feature point may be text or the like showing information such as a place name in the captured image.

[0121] Exemplarily, the feature point identifier may be a name identifier of a reference object indicated by the feature point, such as a company name, a floor identifier, etc.

[0122] In some embodiments, Figure 4 As shown, in the above step 2012, determining the starting position information according to the feature point includes:

[0123] 301: Determine relative position information between the display device and a reference object indicated by the feature point;

[0124] 302: Determine first position information of the feature point in a preset map;

[0125] 303: Determine the starting position information of the starting position in the preset map according to the relative position information and the first position information.

[0126] In some embodiments, in the above step 301, if Figure 5 As shown, the determining the relative position information between the display device and the reference object indicated by the feature point includes:

[0127] 401: Determine horizontal position information and vertical position information of the display device in the scene;

[0128] 402: Establishing a spatial coordinate system according to the horizontal position information and the vertical position information;

[0129] 403: Determine first coordinates and second coordinates of the display device and the reference object indicated by the feature point in the space coordinate system respectively;

[0130] 404: Determine relative position information between the display device and the reference object indicated by the feature point according to the first coordinate and the second coordinate.

[0131] Through the real-time positioning and mapping SLAM algorithm, the pixel value of the feature point in the scene image is identified, and the distance between the reference object indicated by the feature point and the display device is determined. According to the distance between the reference object and the display device, the reference object and the display device are represented in the same coordinate system, and the relative position information between the display device and the reference object indicated by the feature point is determined.

[0132] Specifically, taking the ground in the scene image as a reference and combining the acquisition parameters of the display device when acquiring the scene image, the horizontal position information and the vertical position information of the display device in the scene are determined; based on the horizontal position information and the vertical position information, a spatial coordinate system is established, and the first coordinate of the display device in the spatial coordinate system is determined; the spatial coordinate system indicates the world coordinate system of the display device in the scene.

[0133] Here, the acquisition parameters include but are not limited to the angle, focal length, etc. when the display device acquires the scene image.

[0134] Through the real-time positioning and mapping SLAM algorithm, the distance relationship between the reference object indicated by the feature point and the display device is determined according to the collected image; the second coordinate of the reference object in the spatial coordinate system is determined according to the distance relationship between the reference object and the display device and the first coordinate; and the relative position information between the display device and the reference object indicated by the feature point is determined according to the first coordinate and the second coordinate.

[0135] Before the above step 302, the method further includes: reading the preset map;

[0136] The preset map is a pre-stored map; the preset map can be read from a local database of the display device or from a database of a server.

[0137] In some embodiments, the preset map can be modified or created by an administrator and saved to a local database or a server database;

[0138] In other embodiments, the preset map can be shared with multiple users through a server, or can be shared by a user with other users. This can provide conditions for establishing a large-scale map database.

[0139] In step 302, first position information of the feature point in a preset map is determined;

[0140] Specifically, the feature point identifier of the feature point is used as an index to determine the first position information of the feature point in the preset map.

[0141] Exemplarily, the first position information may be the latitude and longitude coordinates of the reference object indicated by the feature point, or may be the three-dimensional coordinates of the reference object indicated by the feature point in the preset map.

[0142] In the above step 303, the starting position information of the starting position in the preset map is determined according to the relative position information and the first position information.

[0143] In some embodiments, the position information of the display device in the preset map, that is, the starting position information of the starting position in the preset map, is determined based on the relative position relationship between the display device and the reference object and the first position information.

[0144] In the above step 101, generating the target path according to the starting position information and the destination position information includes: generating the target path in the preset map according to the starting position information, the destination position information and the path selection strategy.

[0145] In some embodiments, when there are multiple paths from the starting position indicated by the starting position information and the target position indicated by the target position information, a suitable target path may be selected from the multiple paths according to a path selection strategy.

[0146] In the embodiment of the present disclosure, the path selection strategy includes: selecting a path according to the total length of the target path and / or the congestion condition of the target path.

[0147] In another embodiment, the path selection strategy may also be the time taken to reach the target path, or the road conditions in the target path, such as main roads, small roads, etc.

[0148] When the display device is applied to indoor navigation, the path selection strategy may also be the number of stairs required to be climbed in the target path, etc.

[0149] The path selection strategy may also be customized by the user, which is not specifically limited here.

[0150] Specifically, through the path planning algorithm, according to the starting position information, the destination position information and the path selection strategy, a preset number of alternative paths in the preset map are calculated; the alternative path can be one or more.

[0151] Exemplarily, the path planning algorithm includes but is not limited to: direct search A* (A-star) algorithm, Dijkstra algorithm, dynamic direct search (Dynamic A-star) algorithm, etc.

[0152] In this embodiment, the processing module of the display device determines the target path in the preset map according to the starting position information, the destination position information and the path selection strategy through a direct search algorithm.

[0153] In some embodiments, exemplarily, when there are multiple alternative paths, the processing module of the display device determines at least two alternative paths from the starting location to the destination location in a preset map, and recommends a path that meets the user's preferences among the at least two alternative paths to the user based on the user's pre-set preferences corresponding to the path selection strategy, and uses it as the target path.

[0154] Exemplarily, when there are multiple alternative paths, the processing module of the display device combines the path selection strategy to determine at least two alternative paths from the starting location to the destination location in the preset map, and the user selects one of the at least two alternative paths as the target path.

[0155] In some other embodiments, when there is only one candidate path, the candidate path is used as the target path.

[0156] Exemplarily, the path selection strategy may be that the total length of the target path is the shortest, and the display device determines the path with the shortest total length from the starting position to the destination position in the preset map through a direct search algorithm, and uses it as the target path.

[0157] Exemplarily, the path selection strategy may be that the target path is not congested, and the display device determines, through a direct search algorithm, a non-congested path among the alternative paths from the starting position to the destination position in the preset map, and uses it as the target path.

[0158] After the above step 103, the method further includes:

[0159] The preset map is updated in real time according to the collected images.

[0160] In the embodiment of the present disclosure, after the target path is generated according to the starting position information and the destination position information, the processing module of the display device generates a three-dimensional image for navigation according to the target path, and projects the three-dimensional image to the eyes of the target object. In this way, the processing module is set in the display device, and there is no need to use an external camera to collect images. The reliance on external cameras is small, and its scope of use is wider while reducing the cost of use. In addition, the processing module in the display device directly processes data without the need for data transmission with external devices, so data processing can be achieved more quickly, and three-dimensional image navigation can be displayed to the target object in real time.

[0161] The display method provided by the present disclosure is further described below through specific embodiments.

[0162] The present disclosure provides a display method. Figure 6 and Figure 7 All of them are flow charts of the display method, and the specific process of the method may include the following steps:

[0163] In the prior art, when using mixed reality technology for navigation, it is usually necessary to use multiple external cameras to capture images and combine them with the video of the mobile camera on the head-mounted display device to calculate the positioning. However, since the surrounding environment is displayed differently in the camera under different hardware and light, the recognized environmental features are also different in different devices. The data from multiple cameras cannot be used universally and have low reusability. The display method provided in the embodiment of the present disclosure only involves the camera of the display device itself.

[0164] Step 1: Turn on the display device;

[0165] The display device here may be a head-mounted display device for augmented reality (AR), virtual reality (VR), or mixed reality (MR).

[0166] Step 2: Determine the starting position information;

[0167] Step 2.1: Capture a scene image of the scene where the display device is located through a camera;

[0168] Specifically, the collected scene image is calibrated so that the scene image includes the ground surface of the scene where the human eye is currently located; the scene image here is a depth image.

[0169] The camera collects scene images of the surrounding scenes in real time.

[0170] Step 2.2: Determine the relative position information between the reference object and the display device in the scene through the SLAM algorithm;

[0171] The SLAM algorithm platform here is implemented by the ARcore framework.

[0172] Step 2.2.1: Identify the scene image and determine feature points, i.e., reference objects, in the scene;

[0173] Step 2.2.2: Determine the world coordinates of the display device;

[0174] Specifically, taking the ground of the scene currently located by the human eye as a reference and combining the acquisition parameters when the display device acquires the scene image, the horizontal position information and vertical position information of the display device in the scene are determined; here, the acquisition parameters include but are not limited to the angle, focal length, etc. when the display device acquires the scene image.

[0175] Establishing a spatial coordinate system according to the horizontal position information and the vertical position information;

[0176] Determine a first coordinate of the display device in the spatial coordinate system; the first coordinate here is the world coordinate of the display device.

[0177] Step 2.2.3: Determine the world coordinates of the reference object;

[0178] Specifically, determining the distance between the reference object indicated by the feature point and the display device according to the pixel value of the feature point in the acquired image;

[0179] determining a second coordinate of the reference object in the spatial coordinate system according to the distance between the reference object and the display device and the first coordinate;

[0180] Relative position information between the display device and a reference object indicated by the feature point is determined according to the first coordinate and the second coordinate.

[0181] Step 2.3: Implementing an optical character recognition (OCR) algorithm through the MLKit framework to identify feature point identifiers corresponding to feature points in the scene image;

[0182] Here, the feature point identifier may be a name identifier of a reference object indicated by the feature point, such as a company name, a floor identifier, etc.; or may be a number artificially set for the reference object.

[0183] Step 2.4: Read the preset map and determine the starting position information according to the feature point identifier.

[0184] Step 2.4.1: The preset map may be obtained from a local database of the display device or from a database of a server.

[0185] It should be noted that the operations performed on the preset map are different depending on the user's permissions.

[0186] When the user authority is an administrator, a map can be created or modified; after the administrator modifies the map, the display device automatically saves the modified preset map and imports it into the database.

[0187] When the user authority is that of a common user, the preset map can be read from the database, the preset map can be shared with other users, or the preset map can be uploaded to the server after the navigation is completed.

[0188] Step 2.4.2: Determine the position information of the reference object indicated by the feature point in the preset map according to the feature point identifier;

[0189] Specifically, the feature point identifier of the feature point is used as an index to determine the first position information of the reference object in the preset map, that is, the first position information of the reference object in the preset map.

[0190] The first position information may be the latitude and longitude coordinates of the reference object indicated by the feature point, or may be the world coordinates of the reference object indicated by the feature point in the preset map.

[0191] Step 2.4.3: Determine the starting position information of the starting position in the preset map based on the relative position information between the display device and the reference object, and the first position information of the reference object in the preset map.

[0192] Step 3: Determine the destination location information;

[0193] Specifically, the user can select the destination by himself, specifically, can input the destination by voice, or select the destination from the popular destination options provided by the application, or search for the destination by manual search.

[0194] The destination location information may also be automatically obtained from a calendar or memo set by the user. For example, if the user sets an itinerary in the calendar, the method automatically obtains the destination location information in the itinerary at a preset time according to the set itinerary.

[0195] Step 4: Generate a target path based on the starting location information and the destination location information;

[0196] Step 4.1: Determine the path selection strategy;

[0197] Specifically, the path selection strategy includes but is not limited to at least one of the following: the total length of the target path; the congestion condition of the target path; the time taken for the target path; the road condition in the target path;

[0198] Step 4.2: Generate the target path in the preset map according to the starting position information, the destination position information and the path selection strategy.

[0199] Specifically, by directly searching the A* (A-star) algorithm, a preset number of alternative paths in the preset map are calculated according to the starting position information, the destination position information and the path selection strategy; the alternative path may be one or more.

[0200] Exemplarily, when there are multiple alternative paths, the processing module of the display device combines the path selection strategy to determine at least two alternative paths from the starting location to the destination location in the preset map, and recommends the path of the at least two alternative paths that best meets the user's preferences based on the user's pre-set preferences, and uses it as the target path.

[0201] Exemplarily, when there are multiple alternative paths, the processing module of the display device combines the path selection strategy to determine at least two alternative paths from the starting location to the destination location in the preset map, and the user selects one of the at least two alternative paths as the target path.

[0202] When there is only one candidate path, the candidate path is used as the target path.

[0203] Exemplarily, the path selection strategy may be that the total length of the target path is the shortest, and the display device determines the path with the shortest total length from the starting position to the destination position in the preset map through a direct search algorithm, and uses it as the target path.

[0204] Step 5: Generate a three-dimensional image for navigation and project it to the user's eyes;

[0205] Generally, a map image contains entity points, path points, and connection points, and these three types of points constitute a map;

[0206] In this embodiment, the three-dimensional image at least includes the navigation direction. In this way, when the three-dimensional image is projected onto the eyes of the target object, only the navigation direction is displayed to indicate that the target object has arrived at the destination, making the navigation process more concise and clear without affecting the target object's driving vision.

[0207] Step 6: The user follows the navigation instructions of the projected 3D image and arrives at the destination.

[0208] After the user arrives at the destination, the target path can be uploaded to the server database for sharing with other users; the target path can also be shared with other users to help other users reach the destination.

[0209] Figure 8 is a structural block diagram of a display device according to an exemplary embodiment. Figure 8 The display device 800 may include: a generating module 810, a processing module 820 and a projection module 830, wherein:

[0210] A generating module 810, for generating a target path according to the starting location information and the destination location information;

[0211] A processing module 820, configured to generate a three-dimensional image for navigation according to the target path;

[0212] The projection module 830 is used to project the three-dimensional image to the eyes of the target object.

[0213] In some embodiments, the apparatus further comprises:

[0214] A determination module 840, configured to determine the starting position information;

[0215] The determination module 840 is specifically used to: identify the scene image captured by the display device to obtain the starting position information; or determine the starting position information based on a positioning function.

[0216] In some embodiments, the apparatus further comprises:

[0217] The calibration module 850 is used to calibrate the scene image until the scene image includes the ground of the scene where the human eye is currently located.

[0218] In some embodiments, the determination module 840 is further specifically used to: identify feature points in the scene image; and determine the starting position information according to the feature points.

[0219] In some embodiments, the determining module 840 is further configured to:

[0220] Determine the relative position information between the display device and the reference object indicated by the feature point; determine the first position information of the feature point in the preset map; and determine the starting position information of the starting position in the preset map based on the relative position information and the first position information.

[0221] In some embodiments, the determining module 840 is further configured to:

[0222] Determine the horizontal position information and vertical position information of the display device in the scene; establish a spatial coordinate system based on the horizontal position information and the vertical position information; respectively determine the first coordinate and the second coordinate of the display device and the indicated reference object in the spatial coordinate system; determine the relative position information between the display device and the reference object indicated by the feature point based on the first coordinate and the second coordinate.

[0223] In some embodiments, the generating module 810 is specifically used to:

[0224] The target path is generated in the preset map according to the starting position information, the destination position information and a path selection strategy.

[0225] In some embodiments, the path selection strategy includes at least one of the following:

[0226] The total length of the target path;

[0227] The congestion condition of the target path.

[0228] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0229] In an exemplary embodiment, the generation module 810, the processing module 820, the projection module 830, the determination module 840 and the calibration module 850, etc. can be implemented by one or more central processing units (CPU), graphics processing units (GPU), baseband processors (BP), application specific integrated circuits (ASIC), DSPs, programmable logic devices (PLD), complex programmable logic devices (CPLD), field programmable gate arrays (FPGA), general-purpose processors, controllers, microcontrollers (MCU), microprocessors, or other electronic components to execute the aforementioned method.

[0230] Fig. 9 is a block diagram of a display device 900 according to an exemplary embodiment. For example, the device 900 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0231] Reference Fig. 9 , the device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0232] The processing component 902 generally controls the overall operation of the device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0233] The memory 904 is configured to store various types of data to support operations on the device 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0234] The power supply component 906 provides power to the various components of the device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.

[0235] The multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0236] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the device 900 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0237] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0238] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the device 900. For example, the sensor assembly 914 can detect the open / closed state of the device 900, the relative positioning of components, such as the display and keypad of the device 900, and the sensor assembly 914 can also detect the position change of the device 900 or a component of the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration / deceleration of the device 900, and the temperature change of the device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0239] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and other devices. The device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0240] In an exemplary embodiment, the apparatus 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0241] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the device 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0242] After the instructions are executed by the processor, the following operations can be performed:

[0243] Generate a target path according to the starting location information and the destination location information;

[0244] The processing module of the display device generates a three-dimensional image for navigation according to the target path;

[0245] The three-dimensional image is projected into the eyes of the target object.

[0246] It is understandable that the method further comprises:

[0247] Determining the starting position information;

[0248] The determining the starting position information includes:

[0249] Identify the scene image captured by the display device to obtain the starting position information;

[0250] or,

[0251] The starting position information is determined based on a positioning function.

[0252] It can be understood that before the display device captures the scene image, the method further includes:

[0253] The scene image is calibrated until the scene image includes the ground of the scene where the human eye is currently located.

[0254] It can be understood that the identifying the scene image captured by the display device to obtain the starting position information includes:

[0255] Identifying feature points in the scene image;

[0256] The starting position information is determined according to the feature points.

[0257] It can be understood that determining the starting position information according to the feature point includes:

[0258] Determining relative position information between the display device and a reference object indicated by the feature point;

[0259] Determining first position information of the feature point in a preset map;

[0260] The starting position information of the starting position in the preset map is determined according to the relative position information and the first position information.

[0261] It can be understood that the determining of the relative position information between the display device and the reference object indicated by the feature point includes:

[0262] Determine horizontal position information and vertical position information of the display device in the scene;

[0263] Establishing a spatial coordinate system according to the horizontal position information and the vertical position information;

[0264] respectively determining first coordinates and second coordinates of the display device and the indicated reference object in the spatial coordinate system;

[0265] Relative position information between the display device and a reference object indicated by the feature point is determined according to the first coordinate and the second coordinate.

[0266] It can be understood that the generating of the target path according to the starting position information and the destination position information includes:

[0267] The target path is generated in the preset map according to the starting position information, the destination position information and a path selection strategy.

[0268] It can be understood that the path selection strategy includes at least one of the following:

[0269] The total length of the target path;

[0270] The congestion condition of the target path.

[0271] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the claims.

[0272] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A display method, characterized in that: The method is displayed by a display device, and includes: Generate a target path according to the starting location information and the destination location information; The processing module of the display device generates a three-dimensional image for navigation according to the target path; The three-dimensional image is projected into the eyes of the target object.

2. The method according to claim 1, characterized in that The method further comprises: Determining the starting position information; The determining the starting position information includes: Identify the scene image captured by the display device to obtain the starting position information; or, The starting position information is determined based on a positioning function.

3. The method according to claim 2, characterized in that Before the display device acquires the scene image, the method further includes: The scene image is calibrated until the scene image includes the ground of the scene where the human eye is currently located.

4. The method according to claim 3, characterized in that The identifying the scene image captured by the display device to obtain the starting position information includes: Identifying feature points in the scene image; The starting position information is determined according to the feature points.

5. The method according to claim 4, characterized in that The step of determining the starting position information according to the feature point includes: Determining relative position information between the display device and a reference object indicated by the feature point; Determining first position information of the feature point in a preset map; The starting position information of the starting position in the preset map is determined according to the relative position information and the first position information.

6. The method according to claim 5, characterized in that The determining the relative position information between the display device and the reference object indicated by the feature point includes: Determining the camera coordinates of the display device and the feature points; According to the horizontal position information and the vertical position information in the scene; Establishing a spatial coordinate system according to the horizontal position information and the vertical position information; respectively determining first coordinates and second coordinates of the display device and the indicated reference object in the spatial coordinate system; Relative position information between the display device and a reference object indicated by the feature point is determined according to the first coordinate and the second coordinate.

7. The method according to claim 6, characterized in that The generating of the target path according to the starting position information and the destination position information includes: The target path is generated in the preset map according to the starting position information, the destination position information and a path selection strategy.

8. The method according to claim 7, characterized in that The path selection strategy includes at least one of the following: The total length of the target path; The congestion condition of the target path.

9. A display device, characterized in that: include: A generation module, used for generating a target path according to the starting position information and the destination position information; A processing module, used for generating a three-dimensional image for navigation according to the target path; The projection module is used to project the three-dimensional image to the eyes of the target object.

10. The device according to claim 9, characterized in that The device also includes: A determination module, used to determine the starting position information; The determining module is specifically used for: Identify the scene image captured by the display device to obtain the starting position information; or, The starting position information is determined based on a positioning function.

11. The device according to claim 10, characterized in that The device also includes: The calibration module is used to calibrate the scene image until the scene image includes the ground of the scene currently located by the human eye.

12. The device according to claim 11, characterized in that The determination module is further specifically used for: Identifying feature points in the scene image; The starting position information is determined according to the feature points.

13. The device according to claim 12, characterized in that The determination module is further specifically used for: Determining relative position information between the display device and a reference object indicated by the feature point; Determining first position information of the feature point in a preset map; The starting position information of the starting position in the preset map is determined according to the relative position information and the first position information.

14. The device according to claim 13, characterized in that The determination module is further specifically used for: Determine horizontal position information and vertical position information of the display device in the scene; Establishing a spatial coordinate system according to the horizontal position information and the vertical position information; respectively determining first coordinates and second coordinates of the display device and the indicated reference object in the spatial coordinate system; Relative position information between the display device and a reference object indicated by the feature point is determined according to the first coordinate and the second coordinate.

15. The device according to claim 14, characterized in that The generation module is specifically used for: The target path is generated in the preset map according to the starting position information, the destination position information and a path selection strategy.

16. The device according to claim 15, characterized in that The path selection strategy includes at least one of the following: The total length of the target path; The congestion condition of the target path.

17. An electronic device, characterized in that: include: a memory for storing processor-executable instructions; A processor connected to the memory; The processor is configured to execute the display method provided in any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a computer, enables the computer to perform the display method according to any one of claims 1 to 8.

Images