A method, medium and electronic device for displaying itinerary-related data based on MR

Abstract

The present invention relates to the field of data processing, and in particular to a method, medium and electronic device for displaying itinerary-related data based on MR. It includes: in response to a trigger request for virtual display, performing target recognition processing on image data, generating a background type identifier, a light intensity identifier and a proportion of obstructions corresponding to the image data; obtaining a regional virtual data set corresponding to a preset geographical area to which the current longitude and latitude information belongs based on the current longitude and latitude information obtained; obtaining at least one virtual data subset as a target display set based on the background type identifier, the light intensity identifier and the proportion of obstructions; and displaying using MR technology. In the present invention, since there is no need to buffer virtual data that cannot be used in the current itinerary, the amount of data that the user terminal needs to download and buffer locally can be greatly reduced, thereby increasing the possibility that the user terminal can buffer the complete virtual data locally.

Description

Background Art

[0002] MR (Mixed Reality) is a further development of virtual reality technology. It introduces real scene information into the virtual environment, builds a bridge of interactive feedback information between the virtual world, the real world and the user, and thus enhances the realism of the user experience. The key point of MR technology is to interact with the real world and obtain information in a timely manner. Therefore, its implementation needs to be in an environment that can interact with various things in the real world.

[0003] The interactive feedback capability of MR technology between virtual and reality can display more relevant virtual data based on the current image data to improve interactivity. Before displaying virtual data, the corresponding virtual data needs to be stored locally for subsequent use. Usually, the memory usage of virtual data itself is relatively high. In addition, in some travel scenarios, due to the large changes in the actual scenery along the way, a large amount of virtual data will be required to overlay the corresponding scenery scenes. As a result, the user terminal needs to store a large amount of virtual data locally to ensure that the subsequent virtual data can be displayed completely and normally.

[0004] However, due to the limitation of the remaining memory and cache capacity of the user terminal itself, the corresponding virtual data set may not be downloaded completely, which may result in the virtual data in the itinerary not being displayed completely and normally. Summary of the invention

[0005] In view of the above technical problems, the technical solution adopted by the present invention is:

[0006] According to one aspect of the present invention, a method for displaying itinerary-related data based on MR is provided, the method comprising the following steps:

[0007] During the target travel event running time, image data corresponding to the area to be displayed is obtained;

[0008] In response to a trigger request for virtual display, target recognition processing is performed on the image data to generate a background type identifier, a light intensity identifier, and a proportion of obstructions corresponding to the image data;

[0009] According to the current longitude and latitude information obtained, a regional virtual data set corresponding to a preset geographical area to which the current longitude and latitude information belongs is obtained from a target virtual database corresponding to the target travel event; the regional virtual data set includes at least one type of virtual data subset, and different types of virtual data subsets include different types of virtual data types;

[0010] According to the background type identifier, the illumination intensity identifier and the proportion of the obstruction, at least one virtual data subset is obtained from the regional virtual data set as a target display set;

[0011] Use MR technology to display virtual data in the target display set into the image data;

[0012] The target virtual database includes multiple basic virtual data, which are obtained according to the following steps:

[0013] Before the target trip event is executed, if cache trigger information is obtained, first information collection and processing is performed according to the cache trigger information, where the cache trigger information is information that the corresponding terminal successfully establishes an association relationship with the target trip event; the cache trigger information includes trip track information and trip operation time period information;

[0014] The first information collection process includes:

[0015] According to the itinerary track information, obtain the area identifier corresponding to each associated geographical area; the associated geographical area includes the departure place, destination and en route place of the itinerary;

[0016] Generate a time period identifier corresponding to each associated geographical area according to the itinerary operation time period information;

[0017] Generate a corresponding shielding mark for each associated geographical area according to the meteorological information of each associated geographical area in the corresponding operation sub-period;

[0018] According to the area identifier, time period identifier and occlusion identifier corresponding to each associated geographical area, virtual data with the same identifier is obtained from a preset virtual database and buffered locally as basic virtual data.

[0019] Furthermore, the cache trigger information also includes vehicle riding position information; the target virtual database also includes a plurality of extended virtual data, the extended virtual data being the virtual data remaining after removing the basic virtual data from the virtual data corresponding to the associated geographical area;

[0020] The extended virtual data is obtained as follows:

[0021] Before the target trip event is executed, if cache trigger information is obtained, the second information collection process is performed according to the cache trigger information;

[0022] The second information collection process includes:

[0023] Divide the plurality of extended virtual data into a plurality of preset information collection groups according to preset conditions, each information collection group including at least one extended virtual data; each information collection group has a unique corresponding extended group identifier;

[0024] Generate a seating area identifier of a terminal corresponding to the vehicle seating position information according to the seating area to which the vehicle seating position information belongs;

[0025] According to the mapping relationship between the extended group identifier and the seating area identifier, the extended virtual data in the corresponding information collection group is buffered locally.

[0026] Furthermore, before obtaining, from the target virtual database corresponding to the target trip event, a regional virtual data set corresponding to the preset geographical area to which the current longitude and latitude information belongs, the method further includes:

[0027] If the first terminal or the second terminal detects that the current time is in the corresponding target travel event running time period, then the Bluetooth communication module of the first terminal or the second terminal is activated; the first terminal is a terminal with cached extended virtual data; the second terminal is a terminal with cached basic virtual data;

[0028] The first terminal broadcasts the corresponding first networking information through the Bluetooth communication module; the first networking information includes the extended group identifier corresponding to the information collection group buffered locally by the first terminal and the corresponding first terminal identifier;

[0029] If the second terminal obtains any first networking information, matching the extended group identifier included in the first networking information with the extended group identifier already existing in the second terminal;

[0030] If the matching fails, the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, and saves the first networking information corresponding to the first terminal locally.

[0031] Further, after matching the extended group identifier included in the first networking information with the existing extended group identifier of the second terminal, the method further includes:

[0032] If the match is successful, the second terminal does not respond to the first networking information.

[0033] Furthermore, the first networking information further includes a first ciphertext, which is a ciphertext obtained by encrypting the initial information with the first public key; the first public key corresponds to the vehicle riding position information corresponding to the second terminal;

[0034] The second terminal establishing a Bluetooth connection with the first terminal corresponding to the first terminal identifier includes:

[0035] The second terminal decrypts the first ciphertext using its own corresponding private key to obtain initial information;

[0036] If the initial information is preset verification information, the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier.

[0037] Furthermore, the initial information is a preset symmetric key.

[0038] Furthermore, after the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, the method further includes:

[0039] The Bluetooth communication information between the first terminal and the second terminal is encrypted using a preset symmetric key.

[0040] Furthermore, after the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, the method further includes:

[0041] The extended virtual data buffered in the first terminal is sent to the second terminal.

[0042] According to a second aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for displaying itinerary-related data based on MR is implemented.

[0043] According to a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned MR-based itinerary-related data display method when executing the computer program.

[0044] The present invention has at least the following beneficial effects:

[0045] In the present invention, the departure place, destination and passing places corresponding to the trip can be determined through the trip trajectory information. Then, based on other relevant information that can be obtained, such as the trip operation time period information and weather information, it can be roughly predicted that when the user passes through the corresponding area, the actual scenery corresponding to the time period can be seen and the scenery of those locations is blocked and cannot be seen. Then, based on the actual scenery that the above-mentioned users can see, more targeted virtual data is cached locally. Since there is no need to buffer virtual data that cannot be used in this trip, the amount of data that the user terminal needs to download and buffer locally can be greatly reduced, thereby increasing the possibility that the user terminal can buffer the complete virtual data locally.

[0046] In addition, before the target trip event is executed, if the cache trigger information is obtained, the first information collection process is performed according to the cache trigger information. Then, after the user completes the ticket purchase behavior for a certain trip, the buffering of virtual data can be started. At the same time, since users generally purchase tickets a few days before the trip, the first information collection process can have a longer time, thereby further increasing the possibility that the user terminal can buffer the complete virtual data locally. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 The present invention provides a flowchart of a method for displaying itinerary-related data based on MR in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0050] According to a possible embodiment of the present invention, Figure 1 As shown, a method for displaying itinerary-related data based on MR is provided, which can be applied to travel scenarios, and is more suitable for scenarios of traveling by plane. Specifically, generally, due to the large distance span of traveling by plane, there will be more areas with different types of landscapes passed in one voyage. At the same time, due to the high altitude of the aircraft, a wider field of view can be obtained, and thus more landscape objects can be seen. Therefore, under the influence of the above factors, the user can obtain image data of a variety of different landscapes and objects (such as buildings) during a flight. Then, according to the above-obtained image data, relevant virtual data can be generated through MR technology. These virtual data can be virtual introduction information such as relevant local customs or landscape history of the current area. As a result, the relevant information of the departure place, destination and passage of the route can be displayed and introduced more vividly, thereby improving the passenger's experience during the travel process.

[0051] Specifically, the method comprises the following steps:

[0052] S100: During the target travel event running time period, image data corresponding to the area to be displayed is obtained.

[0053] Specifically, during the flight, passengers can use the camera of their smartphone to shoot downward from the window to obtain image data of the area they are passing through.

[0054] S200: In response to a trigger request for virtual display, target recognition processing is performed on the image data to generate a background type identifier, a light intensity identifier, and a proportion of obstructions corresponding to the image data.

[0055] Since, in order to generate corresponding virtual data on the target in the image data, some relevant information of the object contained in the image data must be determined first. An existing neural network model with image recognition capability can be used to perform target recognition processing to obtain the name and area proportion of each object in the image. Among them, the area of ​​the object can be replaced by the area of ​​the target recognition box (such as the circumscribed rectangular box) of the object.

[0056] Specifically, the image data obtained during the flight of an aircraft can be divided into two categories. One category is that there is no cloud cover in the image, and all the landscape information of the corresponding area below the aircraft is presented, such as mountainous areas, grasslands, forests, urban areas, rural villages, and fields. The other category is that there is cloud cover in the image, and only a part of the area can present the landscape information of the corresponding area below the aircraft, and the rest is completely blocked by clouds.

[0057] Therefore, through target recognition processing, it is possible to determine what objects are in the image, and through target recognition processing, it is also possible to determine whether the current image was obtained during the day or at night. For example, after the image is processed by target recognition, the information generated includes night lighting identification, forest identification and cloud identification. Usually, the object corresponding to the cloud identification is directly identified as an obstruction, and the objects corresponding to the remaining object identifications are directly identified as background objects. At the same time, according to the preset conditions, the background type corresponding to each background object identification can be determined. For example, the first two characters of the forest identification are the corresponding type identification characters 01. The first two characters of the urban area identification are the corresponding type identification characters 02.

[0058] In addition, the area of ​​the cloud is calculated through the target recognition box corresponding to the cloud, and then divided by the total area of ​​the image to obtain the proportion of occlusions.

[0059] S300: According to the acquired current longitude and latitude information, obtain a regional virtual data set corresponding to a preset geographical area to which the current longitude and latitude information belongs from a target virtual database corresponding to the target travel event. The regional virtual data set includes at least one type of virtual data subset, and different types of virtual data subsets include different types of virtual data types.

[0060] When taking a photo, the current longitude and latitude information is obtained. Through the longitude and latitude information, the information of which region the current image belongs to can be determined, and then the regional virtual dataset corresponding to the current image can be determined. For example, if the longitude and latitude information belongs to the longitude and latitude region of Sichuan, the regional virtual dataset is the regional virtual dataset corresponding to Sichuan.

[0061] At the same time, there are many different types of information in a geographical area, such as food, multiple scenic spots, famous buildings, historical figures, cultural symbols, night view buildings, etc. Therefore, each type of information will correspond to a virtual data subset to collect different types of information, and each virtual data subset will correspond to a category identifier.

[0062] S400: Acquire at least one virtual data subset from the regional virtual data set as a target display set according to the background type identifier, the light intensity identifier and the proportion of the obstruction.

[0063] Specifically, when the proportion of obstructions is greater than a preset threshold, the virtual data subset corresponding to the first category identifier is used as the target display set. The first category identifier is the category identifier corresponding to the non-landscape information set, such as the category identifier corresponding to the virtual data subset of food, historical figures and cultural symbols.

[0064] Since the clouds in the image block most of the landscape, it is impossible to present sufficient and complete landscape information in the image. As a result, the virtual data also loses the reference of the real image data and cannot be combined with the real. Therefore, some virtual data that does not need to be combined with the real landscape is directly displayed.

[0065] When the proportion of obstructions is less than or equal to the preset threshold and the light intensity is marked as nighttime light, the virtual data subset corresponding to the second category identifier is used as the target display set. The second category identifier is the category identifier corresponding to the nighttime landscape information set. For example, the category identifier corresponding to the virtual data subset of the iconic nighttime buildings in the urban area.

[0066] When the proportion of obstructions is less than or equal to the preset threshold and the light intensity is marked as daytime light, a virtual data subset corresponding to the category is selected according to the background type as the target display set. If the background type is marked as a forest, the virtual data with the type identification character 01 is added to a set to form the corresponding target display set.

[0067] S500: Using MR technology to display virtual data in the target display set in the image data.

[0068] In the present invention, by processing the target recognition of the image data, the background type identification, the light intensity identification and the proportion of the obstruction corresponding to the image data can be obtained. Then, based on the above information, multiple virtual data that are more in line with the current image data are selected from a large amount of virtual data for display, so as to improve the accuracy of the virtual data display.

[0069] The target virtual database includes multiple basic virtual data, which are obtained according to the following steps:

[0070] S301: Before the target trip event is executed, if cache trigger information is obtained, first information collection and processing is performed according to the cache trigger information, and the cache trigger information is information that the corresponding terminal successfully establishes an association relationship with the target trip event. The cache trigger information includes trip track information and trip operation time period information. Specifically, the cache trigger information can be the ticket information or itinerary information of a trip purchased by the user.

[0071] Since users usually purchase tickets a few days before traveling, and after successfully purchasing tickets, they usually use the same application to view relevant information such as accommodation, food, and attractions. Therefore, the corresponding program for information collection and processing integrated in the same application can be kept open. This allows the first information collection and processing to take a longer time, thereby increasing the possibility that the user terminal can buffer the complete virtual data locally.

[0072] The first information collection process includes:

[0073] S311: Obtaining the area identifier corresponding to each associated geographical area according to the itinerary track information. The associated geographical area includes the departure place, destination and en route place of the itinerary.

[0074] S321: Generate a time period identifier corresponding to each associated geographical area according to the travel operation time period information.

[0075] Taking the route corresponding to a certain flight as an example, since the entire navigation process corresponding to the same flight is basically stable, the corresponding area passed through during the time period is basically the same. Therefore, the time period identifier corresponding to each associated geographical area can be determined based on the historical operation situation. The time period identifier can be composed of the first and last time of the corresponding time period, such as the time period identifier corresponding to 00:00 to 01:30 on March 2, 2022 is: 00.00-01.30.

[0076] S331: Generate a corresponding obstruction mark for each associated geographical area according to the meteorological information of each associated geographical area in the corresponding operating sub-period.

[0077] The meteorological information in this embodiment is mainly cloud information, which can be obtained from the newly available meteorological forecast information. At the same time, since the flight altitude of the aircraft varies during the flight, such as the flight altitude is generally lower than the cloud altitude during the take-off and landing phases, and the flight altitude is generally higher than the cloud altitude during the level flight phase. Therefore, in order to further reduce the amount of data processing in this step, only the meteorological information of the period corresponding to the flight altitude higher than the cloud altitude needs to be processed.

[0078] After the above processing, a corresponding filtering vector will be generated according to the region identifier, time period identifier and occlusion identifier corresponding to each associated geographic area. For example, the filtering vector corresponding to the associated geographic area A is (SC01, 09.40-10.30, 1). The region identifier corresponding to the associated geographic area is SC01, the time period identifier is 09.00-10.30 and the occlusion identifier is 1. Among them, when the occlusion identifier is 1, it means there is cloud occlusion, and when the occlusion identifier is 0, it means there is no cloud occlusion.

[0079] Since flights may cross days, such as taking off at 23:00 on March 2, 2022, and landing at 01:00 on March 3, 2022. At this time, in order to ensure that the time period identifier can be matched normally using the numerical range, when the cross-day phenomenon occurs, 24 will be added to the time after the cross-day to achieve the rationality of the numerical range. The corresponding time period identifier for the above flight is: 23.00-25.00.

[0080] S341: According to the area identifier, time period identifier and occlusion identifier corresponding to each associated geographical area, virtual data with the same identifier is obtained from a preset virtual database and buffered locally as basic virtual data.

[0081] Correspondingly, when making virtual data of a certain area, the corresponding display condition identifier will also be configured for each virtual data. Among them, the display condition identifier includes the above-mentioned area identifier, time period identifier and occlusion identifier. The time period identifier is used to indicate in which time period the virtual data can be displayed, and is mainly used to distinguish the landscape information during the day and at night. The time period identifier corresponding to the virtual data displayed during the day is generally the time period corresponding to the sunrise time to the sunset time, such as 07.00-18.40. The time period identifier corresponding to the virtual data displayed at night is generally the time period corresponding to the sunset time to the sunrise time of the next day. Since it also crosses days, the corresponding time period identifier can be 18.41-30.59.

[0082] The criteria for determining the same identifier in step 341 are as follows:

[0083] When the identifier is a time period identifier, as long as the time ranges represented by the two time period identifiers overlap, they are considered to have the same identifier. Therefore, when the flight period spans daytime and nighttime, the virtual data of the landscapes corresponding to the nighttime and daytime can be collected as basic virtual data, thereby further ensuring the comprehensiveness of the information.

[0084] When the identifier is not a time period identifier, the two identifiers need to be exactly the same to be considered as having the same identifier.

[0085] In the present invention, the departure place, destination and passing places corresponding to the trip can be determined through the trip trajectory information. Then, based on other relevant information that can be obtained, such as the trip operation time period information and weather information, it can be roughly predicted that when the user passes through the corresponding area, the actual scenery corresponding to the time period can be seen and the scenery of those locations is blocked and cannot be seen. Then, based on the actual scenery that the above-mentioned users can see, more targeted virtual data is cached locally. Since there is no need to buffer virtual data that cannot be used in this trip, the amount of data that the user terminal needs to download and buffer locally can be greatly reduced, thereby increasing the possibility that the user terminal can buffer the complete virtual data locally.

[0086] As another possible embodiment of the present invention, the cache trigger information also includes vehicle riding position information. The target virtual database also includes a plurality of extended virtual data, where the extended virtual data is the virtual data remaining after removing the basic virtual data from the virtual data corresponding to the associated geographic area. The vehicle riding position information may be the cabin seat number of the passenger.

[0087] The extended virtual data is obtained as follows:

[0088] S302: before the target travel event is executed, if cache trigger information is obtained, a second information collection process is performed according to the cache trigger information.

[0089] The second information collection process includes:

[0090] S312: Divide the plurality of extended virtual data into a plurality of preset information collection groups according to preset conditions, each information collection group including at least one extended virtual data, and each information collection group having a unique corresponding extended group identifier.

[0091] After the first information collection and processing, there will still be some virtual data that does not meet the screening vector, that is, virtual data with a lower display possibility is inferred based on meteorological conditions and other travel information.

[0092] The preset condition in this step may be a condition of balanced memory usage, that is, the remaining virtual data is divided into a plurality of information collection groups, and the memory usage of each information collection group is substantially the same.

[0093] The preset condition in this step can also be a category segmentation condition, that is, the remaining virtual data is divided into multiple information collection groups according to their corresponding information types, and each information collection group contains only one type of information. Specifically, the corresponding preset condition can be selected according to actual needs.

[0094] S322: Generate a seating area identifier of a terminal corresponding to the vehicle seating position information according to the seating area to which the vehicle seating position information belongs.

[0095] Specifically, since the passengers sitting on the aisle side of the cabin are far away from the portholes and basically cannot capture images of the lower area, the terminals in this area are set as relay terminals to download extended virtual data, so as to provide more virtual data for the terminals near the window side in the future.

[0096] S332: According to the mapping relationship between the extended group identifier and the seating area identifier, the extended virtual data in the corresponding information collection group is buffered locally.

[0097] In addition, in order to improve the success rate of downloading, terminals in different seats will be allowed to download the extended virtual data in the same group. And because the aircraft cabin is distributed in a long strip, in order to ensure the stability of Bluetooth communication in the later stage, it is necessary to shorten the distance of Bluetooth communication between different terminals. Therefore, the area of ​​the middle seats in the long strip distribution can be divided into three sub-areas: front, middle and back, and in each sub-area, terminals on different seats are selected to download extended virtual data of different groups. For example, there are 6 rows of seats in the front sub-area, with two seats in each row, and the extended virtual data is divided into 3 information collection groups. In this case, the terminals of the passengers in the first and fourth rows can download the extended virtual data in the first information collection group; the terminals of the passengers in the second and fifth rows can download the extended virtual data in the second information collection group; the terminals of the passengers in the third and sixth rows can download the extended virtual data in the third information collection group.

[0098] After the second information collection and processing, the terminals in the same flight can download the corresponding virtual data according to the use needs, thereby reducing the amount of data that each terminal needs to download to ensure the integrity of the downloaded data. At the same time, after the information held by different terminals is shared, a complete virtual database can be established, thereby improving the integrity of the virtual data.

[0099] During the flight, the flight is very likely to be delayed, which will cause the flight period to change. At the same time, due to the changing weather conditions, it is very likely that the previously predicted cloud conditions will be very different from the actual cloud conditions during the flight. As a result, the previously downloaded basic virtual data cannot be used. At this time, the extended virtual data shared by other terminals can be displayed to ensure normal virtual data display.

[0100] In this embodiment, the purpose of sharing virtual data between different terminals is achieved by the following method, which specifically includes the following steps:

[0101] Before S300: obtaining, from the target virtual database corresponding to the target trip event, a regional virtual data set corresponding to the preset geographical area to which the current longitude and latitude information belongs, the method further includes:

[0102] S210: If the first terminal or the second terminal detects that the current time is in the corresponding target travel event running time period, then start its own Bluetooth communication module. The first terminal is a terminal that caches the extended virtual data. The second terminal is a terminal that caches the basic virtual data.

[0103] S220: The first terminal broadcasts corresponding first networking information through the Bluetooth communication module. The first networking information includes an extended group identifier corresponding to the information collection group buffered locally by the first terminal and a corresponding first terminal identifier.

[0104] S230: If the second terminal obtains any first networking information, a matching process is performed on the extended group identifier included in the first networking information and the extended group identifier already existing in the second terminal.

[0105] S240: If the matching fails, the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, and saves the first networking information corresponding to the first terminal locally.

[0106] S250: If the match is successful, the second terminal does not respond to the first networking information.

[0107] If the match fails, it means that the second terminal lacks the corresponding extended virtual data in the first terminal, so a Bluetooth connection is established to facilitate the acquisition of the corresponding virtual data from the first terminal when needed later. This operation can ensure that the virtual data will not be concentrated in the same terminal as much as possible, thereby reducing memory usage.

[0108] Since the Internet is not available during the flight phase, Bluetooth connection is used to achieve communication between any two terminals, providing a basis for sharing virtual data.

[0109] Furthermore, the first networking information also includes a first ciphertext, which is the ciphertext obtained after the initial information is encrypted with the first public key. The first public key corresponds to the vehicle riding position information corresponding to the second terminal. Specifically, a corresponding public key and private key will be configured in advance for each seat according to the actual usage scenario, and then a key table for the seats in the entire cabin will be generated. Therefore, before sending the networking information, the first terminal encrypts it with each public key to generate multiple different first ciphertexts, and then broadcasts them.

[0110] Correspondingly, in order to reduce the amount of broadcast information, the same public key and private key may be set for multiple different terminals.

[0111] S240: The second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, including:

[0112] S241: The second terminal decrypts the first ciphertext using its own corresponding private key to obtain initial information.

[0113] S242: If the initial information is preset verification information, the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier. Specifically, the initial information is a preset symmetric key.

[0114] When the first terminal and the second terminal communicate for the first time, that is, when the first networking information is sent, the first ciphertext is also attached for further verification. If a Bluetooth connection needs to be established, the first ciphertext is decrypted by the private key, and the decrypted initial information is verified. The connection can be established only after the verification is passed. Since asymmetric encryption has a higher level of protection, it can prevent the verification information from being stolen, thereby improving the accuracy of the verification and the security of subsequent communication between the first terminal and the second terminal via Bluetooth.

[0115] After S240, the method further includes:

[0116] S243: Using a preset symmetric key, encrypt the Bluetooth communication information between the first terminal and the second terminal.

[0117] Specifically, after the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, the extended virtual data buffered in the first terminal is sent to the second terminal.

[0118] Symmetric encryption can increase the speed of encryption and decryption, and thus increase the speed of subsequent data transmission between two terminals. Since the identity of the terminal has been verified by asymmetric encryption with a higher protection level during the first communication, in the subsequent data transmission process, while taking into account confidentiality, more attention is paid to data transmission speed, so symmetric encryption is selected as the encryption method for subsequent data transmission.

[0119] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0120] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0121] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0122] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0123] The electronic device according to this embodiment of the present invention is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0124] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

[0125] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0126] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0127] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0128] The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0129] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device (e.g., routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0130] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0131] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0132] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0133] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0134] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0135] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0136] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0137] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0138] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for displaying itinerary-related data based on MR, characterized in that: The method comprises the following steps: During the target travel event running time, image data corresponding to the area to be displayed is obtained; In response to a trigger request for virtual display, target recognition processing is performed on the image data to generate a background type identifier, a light intensity identifier, and a proportion of obstructions corresponding to the image data; According to the acquired current longitude and latitude information, a regional virtual data set corresponding to a preset geographical area to which the current longitude and latitude information belongs is acquired from a target virtual database corresponding to the target travel event; the regional virtual data set includes at least one type of virtual data subset, and different types of virtual data subsets include different types of virtual data types; According to the background type identifier, the illumination intensity identifier and the proportion of the obstruction, at least one virtual data subset is obtained from the regional virtual data set as a target display set; Using MR technology to display virtual data in the target display set into the image data; The target virtual database includes a plurality of basic virtual data, and the basic virtual data is obtained according to the following steps: Before the target travel event is executed, if cache trigger information is obtained, first information collection processing is performed according to the cache trigger information, wherein the cache trigger information is information that the corresponding terminal successfully establishes an association relationship with the target travel event; The cache trigger information includes travel trajectory information and travel operation time period information; The first information collection process includes: According to the travel trajectory information, obtain the area identifier corresponding to each associated geographical area; the associated geographical area includes the departure place, destination and en route place of the travel; Generate a time period identifier corresponding to each associated geographical area according to the travel operation time period information; Generate a corresponding shielding mark for each associated geographical area according to the meteorological information of each associated geographical area in the corresponding operation sub-period; According to the area identifier, time period identifier and occlusion identifier corresponding to each associated geographical area, virtual data with the same identifier is obtained from a preset virtual database and buffered locally as basic virtual data.

2. The method according to claim 1, characterized in that The cache trigger information also includes vehicle riding position information; the target virtual database also includes a plurality of extended virtual data, wherein the extended virtual data is virtual data remaining after removing the basic virtual data from the virtual data corresponding to the associated geographical area; The extended virtual data is obtained according to the following steps: Before the target trip event is executed, if cache trigger information is obtained, performing second information collection processing according to the cache trigger information; The second information collection process includes: Divide the plurality of extended virtual data into a plurality of preset information collection groups according to preset conditions, each of the information collection groups including at least one extended virtual data; each of the information collection groups has a unique corresponding extended group identifier; Generate a seating area identifier of a terminal corresponding to the vehicle seating position information according to the seating area to which the vehicle seating position information belongs; According to the mapping relationship between the extended group identifier and the seating area identifier, the extended virtual data in the corresponding information collection group is buffered locally.

3. The method according to claim 2, characterized in that Before obtaining, based on the acquired current longitude and latitude information, a regional virtual data set corresponding to a preset geographical area to which the current longitude and latitude information belongs from a target virtual database corresponding to the target travel event, the method further includes: If the first terminal or the second terminal detects that the current time is in the corresponding target travel event running time period, then the Bluetooth communication module of the first terminal or the second terminal is activated; the first terminal is a terminal with cached extended virtual data; the second terminal is a terminal with cached basic virtual data; The first terminal broadcasts the corresponding first networking information through the Bluetooth communication module; the first networking information includes the extended group identifier corresponding to the information collection group buffered locally by the first terminal and the corresponding first terminal identifier; If the second terminal obtains any of the first networking information, matching the extended group identifier included in the first networking information with the extended group identifier already possessed by the second terminal; If the matching fails, the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, and saves the first networking information corresponding to the first terminal locally.

4. The method according to claim 3, characterized in that After matching the extended group identifier included in the first networking information with the existing extended group identifier of the second terminal, the method further includes: If the match is successful, the second terminal does not respond to the first networking information.

5. The method according to claim 3, characterized in that: The first networking information further includes a first ciphertext, where the first ciphertext is a ciphertext obtained by encrypting the initial information with a first public key; the first public key corresponds to the vehicle riding position information corresponding to the second terminal; The second terminal establishing a Bluetooth connection with the first terminal corresponding to the first terminal identifier includes: The second terminal decrypts the first ciphertext using its own corresponding private key to obtain initial information; If the initial information is preset verification information, the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier.

6. The method according to claim 5, characterized in that The initial information is a preset symmetric key.

7. The method according to claim 6, characterized in that After the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, the method further includes: The preset symmetric key is used to encrypt the Bluetooth communication information between the first terminal and the second terminal.

8. The method according to claim 7, characterized in that After the second terminal establishes a Bluetooth connection with the first terminal corresponding to the first terminal identifier, the method further includes: The extended virtual data buffered in the first terminal is sent to the second terminal.

9. A non-transitory computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for displaying itinerary-related data based on MR is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for displaying itinerary-related data based on MR is implemented.

Images