Interaction method and device, viewing equipment and computer readable storage medium

Abstract

The invention discloses an interaction method and device, viewing equipment and a computer readable storage medium, the method is applied to the viewing equipment in a viewing system, the viewing system further comprises processing equipment, and the method comprises the steps that current pose information of a user is collected; acquiring map data from processing equipment according to the current pose information of the user; displaying the target map according to the map data; and in response to a selection operation of a user on a target object in the target map, displaying associated information corresponding to the target object. In the embodiment of the invention, the user not only can use the viewing equipment to view the scene, but also can use the viewing equipment to view the required related information while using the viewing equipment to view the scene, so that the user experience can be improved.

Description

Technical Field

[0001] This application relates to the field of mixed reality technology, and particularly to an interaction method, device, viewing device and computer-readable storage medium. Background Art

[0002] During travel or play, users can view scenery remotely through viewing devices such as telescopes. When users need to know relevant information during the viewing process, they can obtain the relevant information through methods such as network queries. In the case of network queries, users need to put down the viewing device and then use a mobile device such as a mobile phone to query. However, during the process of using the mobile terminal to query relevant information, users cannot use the viewing device, resulting in a poor user experience. Summary of the Invention

[0003] Embodiments of this application disclose an interaction method, device, viewing device and computer-readable storage medium for improving user experience.

[0004] In a first aspect, embodiments of this application disclose an interaction method, which is applied to a viewing device in a viewing system. The viewing system further includes a processing device, and includes:

[0005] Collect the current pose information of the user;

[0006] Obtain map data from the processing device according to the current pose information;

[0007] Display a target map according to the map data;

[0008] In response to a user's selection operation on a target object in the target map, display associated information corresponding to the target object.

[0009] In a second aspect, embodiments of this application disclose an interaction device, which is applied to a viewing device in a viewing system. The viewing system further includes a processing device, and includes:

[0010] A collection unit for collecting the current pose information of the user;

[0011] An acquisition unit for obtaining map data from the processing device according to the current pose information;

[0012] A display unit for displaying a target map according to the map data;

[0013] The display unit is further configured to display associated information corresponding to the target object in response to a user's selection operation on the target object in the target map.

[0014] In a third aspect, an embodiment of the present application discloses a viewing device, including a processor and a memory. The processor is configured to call a computer program stored in the memory to execute the method disclosed in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application discloses a computer-readable storage medium, on which a computer program or computer instructions are stored. When the computer program or computer instructions are run by a processor, the method disclosed in the first aspect as described above is implemented.

[0016] In a fifth aspect, an embodiment of the present application discloses a computer program product, which includes computer program code. When the computer program code is run by a processor, the above method is executed.

[0017] In the embodiment of the present application, the viewing device collects the current pose information of the user, obtains map data from the processing device according to the current pose information of the user, displays a target map according to the map data, and in response to a selection operation of a target object in the target map, displays associated information corresponding to the target object. It can be seen that the user can not only use the viewing device for viewing, but also view the relevant information corresponding to the target object needed while using the viewing device for viewing, which can improve the user experience. In addition, since the pose information of the user is different, the obtained map data is different, and the displayed maps are also different. Therefore, the viewing device can automatically display different maps for the user according to the pose information of the user, which can improve the flexibility, applicability and effectiveness of viewing, and can further improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0019] Figure 1 is a schematic diagram of a network architecture disclosed in an embodiment of the present application;

[0020] Figure 2 is a schematic flowchart of an interaction method disclosed in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a natural landform icon disclosed in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of the name information of a natural landform disclosed in an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of highlighting buildings disclosed in an embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of highlighting traffic disclosed in an embodiment of the present application;

[0025] Figure 7 It is a schematic diagram of full - screen display disclosed in an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of displaying associated information of mountains and rivers disclosed in an embodiment of the present application;

[0027] Figure 9 It is a schematic diagram of points of interest of urban types disclosed in an embodiment of the present application;

[0028] Figure 10 It is a schematic diagram of points of interest of scenic spot types disclosed in an embodiment of the present application;

[0029] Figure 11 It is a schematic diagram of associated information corresponding to key points of interest disclosed in an embodiment of the present application;

[0030] Figure 12 It is a schematic diagram of associated information corresponding to non - key points of interest disclosed in an embodiment of the present application;

[0031] Figure 13 It is a schematic flowchart of another interaction method disclosed in an embodiment of the present application;

[0032] Figure 14 It is a schematic structural diagram of an interaction device disclosed in an embodiment of the present application;

[0033] Figure 15 It is a schematic structural diagram of a viewing device disclosed in an embodiment of the present application. Detailed implementation manners

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0035] The embodiments of the present application disclose an interaction method, device, viewing device, and computer - readable storage medium for improving user experience. The following will be described in detail respectively.

[0036] To better understand the embodiments of the present application, the network architecture of the present application will be introduced first below.

[0037] Please refer to Figure 1 , Figure 1 It is a schematic diagram of a network architecture disclosed in an embodiment of the present application. AsFigure 1 As shown, the network architecture may include a processing device 101 and multiple viewing devices 102. The processing device 101 may be connected to the multiple viewing devices 102 via a network. The network may be a wide area network, a local area network, or a combination of the two.

[0038] The viewing device 102 can collect the current posture information of the user, and then send the collected current posture information of the user to the processing device 101. After the processing device 101 receives the current posture information of the user from the viewing device 102, it can first determine the user's viewing angle information based on the user's current posture information, and then determine the map data based on the user's viewing angle information, and then send the map data to the viewing device 102. Correspondingly, the viewing device 102 can receive the map data from the processing device 101, and then display the target map based on the map data. The user can select the target object that needs to be interacted with from the displayed target map as needed. After the viewing device 102 detects the user's selection operation on the target object in the target map, it can display the associated information corresponding to the target object in response to the user's selection operation on the target object in the target map.

[0039] A viewing device 102 may include a mixed reality (MR) device, or may include multiple mixed reality devices.

[0040] In the case where a viewing device 102 includes a mixed reality device, the viewing device 102 can display the target map through the mixed reality device, and can display the associated information corresponding to the target object through the mixed reality device.

[0041] When a viewing device 102 includes multiple mixed reality devices, the contents displayed by the multiple mixed reality devices may be the same or different.

[0042] Exemplarily, in the case where one mixed reality device among multiple mixed reality devices is a master mixed reality device and the other mixed reality devices are slave mixed reality devices, the viewing device 102 collects the current posture information of the user using the master mixed reality device, and in response to the user using the master mixed reality device selecting the target object in the target map, the associated information corresponding to the target object is displayed respectively through the multiple mixed reality devices. For example, in the explanation scene, the mixed reality device used by the tour guide can be the master mixed reality device, and the mixed reality devices used by other users can be slave mixed reality devices. The explanation scene can be applicable to any explanation scene, such as explanations between adults, explanations between adults and children, explanations between children, etc.

[0043] Exemplarily, when the current pose information of users using multiple mixed reality devices is the same, the viewing device 102 can obtain map data from the processing device according to the current pose information of the users, and display the associated information corresponding to the target object through the multiple mixed reality devices respectively.

[0044] The selection operations corresponding to different mixed reality devices for using the user can be different, and correspondingly, the selected target objects can be different.

[0045] In some embodiments, different mixed reality devices can communicate with each other. Users can achieve information sharing through the used mixed reality devices.

[0046] In some embodiments, the viewing device is a device that can collect the pose information of the user and communicate with the processing device. Exemplarily, the viewing device can be an MR device.

[0047] In some embodiments, the processing device can be a device or server with processing functions and communication functions. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0048] It should be understood that Figure 1 is an exemplary illustration of the network architecture of the present application, and does not limit the network architecture of the present application.

[0049] Please refer to Figure 2 , Figure 2 is a schematic flowchart of an interaction method disclosed in an embodiment of the present application. Among them, the interaction method can be applied to the viewing device in the viewing system, and the viewing system can further include a processing device. As Figure 2 shown, the interaction method can include the following steps.

[0050] 201. Collect the current pose information of the user.

[0051] The viewing device can collect the current pose information of the user. Specifically, the viewing device can collect the current pose information of the user in real time, or at regular intervals, or periodically. The current pose information of the user is the pose information of the current user.

[0052] The viewing device can include a pose collection device. The viewing device can collect the current pose information of the user through the pose collection device.

[0053] The pose acquisition device may include a positioning module and a visual tracking module. The user's current pose information may include the user's current position information and current orientation information. The user's current orientation information is the orientation information of the user's current face, that is, the gaze direction information of the user's current eyes.

[0054] The viewing device may collect the user's current position information through the positioning module. The positioning module may be a device with positioning functions such as the Global Positioning System (GPS), Beidou Navigation System, etc.

[0055] The viewing device may collect the user's current orientation information through the visual tracking module. The visual tracking module may include one or more of a geomagnetic sensor, a gyroscope, a gyroscope combined with a binocular camera, and an acceleration sensor, etc.

[0056] In some embodiments, the viewing device may first detect whether the viewing device is being used by the user. When it is detected that the viewing device is being used by the user, the user's current pose information may be collected. When it is detected that the viewing device is not being used by the user, it indicates that there is no user using the viewing device currently, and step 201 may not be executed, which can avoid unnecessary operations and thus reduce power consumption.

[0057] In one case, the viewing device may detect whether the viewing device is being used by the user through the visual tracking module. In another case, the viewing device may include an outer red line device, and the viewing device may detect whether the viewing device is being used through the outer red line device. In yet another case, the viewing device may include a camera, and the viewing device may detect whether the viewing device is being used through the camera. Specifically, the viewing device may collect an image through the camera, and then may identify whether there is a user in the image. When it is identified that there is a user in the image, it may be determined that the viewing device is being used by the user. When it is identified that there is no user in the image, it may be determined that the viewing device is not being used by the user.

[0058] It should be understood that the above is an exemplary description of detecting whether the viewing device is being used by the user, and does not limit the detection of whether the viewing device is being used by the user.

[0059] 202. Obtain map data from the processing device according to the user's current pose information.

[0060] After the viewing device collects the user's current pose information, it may obtain map data from the processing device according to the user's current pose information. The map data may include the data required to generate a map.

[0061] In some embodiments, after the viewing device acquires the current pose information of the user, it may first determine whether the current pose information of the user is the same as the previous pose information of the user. When it is determined that the current pose information of the user is the same as the previous pose information of the user, it indicates that the pose information of the user has not changed, and step 202 may not be executed, which can avoid unnecessary processing procedures and thus reduce power consumption. In addition, the number of interactions between the viewing device and the processing device can be reduced, communication resources can be saved, and the network load can be reduced. When it is determined that the current pose information of the user is different from the previous pose information of the user, it indicates that the pose information of the user has changed, and map data can be obtained from the processing device according to the current pose information of the user, so as to update the viewing map of the user as the pose information of the user changes, thereby ensuring the flexibility, applicability, and effectiveness of the user's viewing. The previous pose information of the user is the pose information of the user collected last time.

[0062] In some embodiments, after the viewing device acquires the current pose information of the user, it may first determine whether a map is currently displayed. When a map is currently displayed, it may continue to determine whether the pose information corresponding to the displayed map is the same as the current pose information of the user. When the pose information corresponding to the displayed map is different from the current pose information of the user, it indicates that the pose information of the user has changed, and map data can be obtained from the processing device according to the current pose information of the user. Different pose information of the user corresponds to different map data. Therefore, when the pose information of the user changes, the corresponding map data can be obtained according to the current pose information of the user, which can ensure that different viewing maps can be seen by the user and different objects can be interacted with by the user as the pose information of the user is different, improving the flexibility, applicability, and effectiveness of the interaction and enhancing the user experience. In addition, only when the pose information of the user changes can the corresponding map data be obtained according to the current pose information of the user, which can avoid obtaining map data when the pose information of the user has not changed, reduce unnecessary processing procedures, and thus reduce the power consumption of the viewing device. Further, the number of interactions between the viewing device and the processing device can be reduced, communication resources can be saved, and the network load can be reduced.

[0063] When no map is currently displayed, map data can be directly obtained from the processing device according to the current pose information of the user so that the user can view the scene through the viewing device. When the pose information corresponding to the displayed map is the same as the current pose information of the user, it indicates that the pose information of the user has not changed, and step 202 may not be executed, which can avoid unnecessary processing procedures and thus reduce power consumption. In addition, the number of interactions between the viewing device and the processing device can be reduced, communication resources can be saved, and the network load can be reduced.

[0064] Specifically, the viewing device can send the current pose information of the user to the processing device. Correspondingly, the processing device can receive the current pose information of the user from the viewing device, and then can determine the viewing angle information of the user based on the current pose information of the user, can determine the map data based on the viewing angle information of the user, and can send the map data to the viewing device. Correspondingly, the viewing device can receive the map data from the processing device.

[0065] The processing device can determine the heading angle of the viewing device according to the current orientation information of the user, and can determine the viewing angle information of the user, that is, the viewing angle information of the viewing device, according to the current position information of the user and the heading angle of the viewing device.

[0066] 203. Display the target map according to the obtained map data.

[0067] After the viewing device obtains the map data from the processing device according to the current pose information of the user, it can generate a target map according to the map data, and then can display the target map.

[0068] 204. In response to the user's selection operation on the target object in the target map, display the associated information corresponding to the target object.

[0069] After the viewing device displays the target map according to the obtained map data, when the user needs to obtain the associated information corresponding to the target object in the target map, the user can select the target object in the target map in various ways. Correspondingly, the viewing device can detect the user's selection operation on the target object in the target map, and in response to the user's selection operation on the target object in the target map, display the associated information corresponding to the target object.

[0070] The target object can be any object in the target map. The target object can be a building, or a natural landform, or transportation. Buildings can include various artificially constructed objects. Natural landforms can include mountains, green spaces, wetlands, lakes, rivers, etc. Transportation can include roads, tracks, subways, bridges, etc. Roads can include national highways, provincial highways, national roads, urban expressways, provincial roads, main arterial county roads, scenic area roads, secondary arterial roads, pedestrian paths, tour bus routes, etc.

[0071] The associated information corresponding to the target object is the information associated with the target object, and can include one or more of the name information of the target object, the text information corresponding to the target object, and the video information corresponding to the target object.

[0072] The name information of the target object may include the name of the target object. The name information of the target object may also include the category icon of the target object. The category icon of the target object may be a building icon, a natural landform icon, or a transportation icon. The transportation icon may be a road icon, a rail / subway icon, or a bridge icon. The natural landform icon may be a mountain type icon or a water type icon. Exemplarily, please refer to Figure 3 , Figure 3 which is a schematic diagram of a natural landform icon disclosed in an embodiment of the present application. As Figure 3 shown, when the natural landform is mountains, greenlands, or wetlands, the natural landform icon is a mountain type icon; when the natural landform is water type natural landforms such as lakes and rivers, the natural landform icon is a water type icon.

[0073] It should be understood that Figure 3 is an exemplary description of the mountain type icon and the water type icon, and does not limit them.

[0074] When the target object is a natural landform, the name information of the target object may also include a guiding line. Exemplarily, please refer to Figure 4 , Figure 4 which is a schematic diagram of the name information of a natural landform disclosed in an embodiment of the present application. As Figure 4 shown, the name information of the natural landform may include a guiding line and a name panel, and the name panel may include a natural landform icon and the name of the natural landform.

[0075] It should be understood that Figure 4 is an exemplary description of the name information of the natural landform, and does not limit it.

[0076] The text information corresponding to the target object is the text information associated with the target object, and may include one or more of the information related to the target object such as the name of the target object, the location information of the target object, the distance between the viewing device and the target object, the travel mode, and the introduction information of the target object. The video information corresponding to the target object is the video information associated with the target object.

[0077] In some embodiments, the viewing device may, in response to the user's selection operation on the target object in the target map, highlight the target object, and then may, in response to the user's confirmation operation on the target object, display the associated information corresponding to the target object.

[0078] When the user needs to view or watch the associated information corresponding to the target object, the user can first select the target object in the target map. Accordingly, the viewing device can detect the user's selection operation on the target object in the target map, and can, in response to the user's selection operation on the target object in the target map, highlight the target object. When the selected target object is the object the user needs, the user can continue to perform a confirmation operation on the target object. Accordingly, the viewing device can detect the user's confirmation operation on the target object, and can, in response to the user's confirmation operation on the target object, display the associated information corresponding to the target object. When the selected target object is an object the user does not need, the user can refrain from performing a confirmation operation on the target object, which can avoid the situation of the user's misoperation, thereby improving the user experience.

[0079] Highlighting the target object can be highlighting the target object in bold, or can be blurring the target object, or can be floating the target object, or can be full-screen displaying the target object, or can be a display method different from other objects in the target map, so that it can be reflected that the user has selected the target object through different display methods. Full-screen displaying the target object can be understood as full-screen displaying the target object on the target map.

[0080] Depending on the type of the target object, the way to highlight the target object can be different.

[0081] Exemplarily, when the target object is a building, highlighting the target object can be highlighting the target object in bold; when the target object is traffic, highlighting the target object can be blurring the target object; when the target object is natural landforms, highlighting the target object can be floating the target object.

[0082] It should be understood that the above is an exemplary description of the highlighting methods for different types of objects, and is not intended to limit them.

[0083] Exemplarily, please refer to Figure 5 , Figure 5 which is a schematic diagram of highlighting a building disclosed in an embodiment of the present application. Exemplarily, please refer to Figure 6 , Figure 6 which is a schematic diagram of highlighting traffic disclosed in an embodiment of the present application. Exemplarily, please refer to Figure 7 , Figure 7 which is a schematic diagram of full-screen display disclosed in an embodiment of the present application. Exemplarily, please refer to Figure 8 , Figure 8 which is a schematic diagram of highlighting mountains and rivers disclosed in an embodiment of the present application. As Figure 8As shown, the mountains and rivers can have a light carpet special effect. Only the edges of the turns of the mountain body can be highlighted, and corresponding gradient effects can be displayed according to the height and depth of the mountains and rivers. The suspended part is the natural model entity contour and particle filling effect of the mountains and rivers.

[0084] The viewing device can include two exploration modes. One exploration mode is to determine the display method according to the type of the target object, and the other is full-screen display.

[0085] In the case where the user selects full-screen display, the viewing device can display the target object in full screen. In the case where the user does not select full-screen display, the viewing device can first determine the type of the target object, and then highlight the target object according to the type of the target object.

[0086] The user can select the target object in the target map by voice, or can select the target object in the target map by mechanical buttons and / or function buttons on the viewing device, or can also select the target object in the target map through the visual tracking module, or can also select the target object in the target map by combining two or three of the above methods.

[0087] In some embodiments, in the case where the user needs to view or watch the associated information corresponding to the target object in the target map, the user can output voice information including "information of the target object". Correspondingly, the viewing device can collect the voice information through the voice collection device, then can extract the information of the target object from the voice information, can obtain the associated information corresponding to the target object from the map data according to the information of the target object, and then can display the associated information corresponding to the target object. The information of the target object can be the name of the target object, or other information that can uniquely identify the target object.

[0088] In some embodiments, in the case where the user needs to view or watch the associated information corresponding to the target object in the target map, the user can select the target object in the target map by mechanical buttons and / or function buttons on the viewing device. Exemplarily, the user can move the cursor to the position of the target object in the target map by mechanical buttons and / or function buttons on the viewing device. The viewing device can highlight the target object in response to the user's selection operation. The user can perform a confirmation operation on the target object by mechanical buttons or function buttons. The viewing device can obtain the associated information corresponding to the target object from the map data in response to the user's confirmation operation on the target object, and then can display the associated information of the target object.

[0089] In some embodiments, the viewing device can detect the user's gaze through the visual tracking module. When the viewing device detects, through the visual tracking module, that the user is gazing at a target object in the target map, it indicates that the user has selected the target object, and the viewing device can highlight the target object. When the user needs to view or check the associated information corresponding to the target object, the user can confirm the highlighted target object. Accordingly, the viewing device can detect the user's confirmation operation on the target object, and in response to this confirmation operation, obtain the associated information corresponding to the target object from the map data, and then display the associated information corresponding to the target object.

[0090] In one case, after the viewing device highlights the target object, it can record the duration of the user's gaze on the target object. Before the duration of the user's gaze on the target object is less than the preset duration, if the user's gaze object changes, it indicates that the user does not need to view the associated information corresponding to the target object, and the associated information corresponding to the target object may not be displayed, which can avoid the situation of user misoperation and thus improve the user experience. When the duration of the user's gaze on the target object is equal to the preset duration, it indicates that the user needs to view the associated information corresponding to the target object, and it can be determined that the user's confirmation operation on the target object is detected. The associated information corresponding to the target object can be obtained from the map data, and then the associated information corresponding to the target object can be displayed.

[0091] In another case, after the viewing device highlights the target object, the user can perform a confirmation operation through a mechanical button or a function button on the viewing device. Exemplarily, the viewing device can include a confirmation button. When the viewing device detects the user's operation on the confirmation button, it can be determined that the user's confirmation operation on the target object is detected. The associated information corresponding to the target object can be obtained from the map data, and then the associated information corresponding to the target object can be displayed.

[0092] It can be seen that the user can not only use the viewing device for viewing the scenery, but also use the viewing device to view the required relevant information while viewing the scenery, which can improve the user experience.

[0093] When the associated information corresponding to the target object only includes the name of the target object, the viewing device can display the name of the target object in the area where the target object is located, or in an area whose distance from the area where the target object is located is less than the preset distance, or in a preset area. The name of the target object can be displayed in a floating manner, or in a highlighted manner, or in other ways.

[0094] When the associated information corresponding to the target object only includes the text information corresponding to the target object, the viewing device can display the text information corresponding to the target object in the area where the target object is located, or can display the text information corresponding to the target object in an area whose distance from the area where the target object is located is less than a preset distance, or can also display the text information corresponding to the target object in a preset area. The displayed text information corresponding to the target object can be displayed in a floating manner, or can be displayed in a highlighted manner, or can also be displayed in other ways.

[0095] When the associated information corresponding to the target object only includes the video information corresponding to the target object, the viewing device can display the video information corresponding to the target object in the area where the target object is located, or can display the video information corresponding to the target object in an area whose distance from the area where the target object is located is less than a preset distance, or can also display the video information corresponding to the target object in a preset area. Displaying the video information corresponding to the target object can be directly playing the video information corresponding to the target object, or can be displaying the video file of the video information corresponding to the target object. When what is displayed is the video file of the video information corresponding to the target object and the user needs to view the video information corresponding to the target object, the user can perform a playing operation on the video file corresponding to the target object in the target map in various ways. The viewing device can detect the playing operation of the user on the video file corresponding to the target object in the target map and can respond to the playing operation to play the video information corresponding to the target object.

[0096] When the associated information corresponding to the target object includes the name of the target object and the text information (or video information) corresponding to the target object, the viewing device can display the name of the target object and the text information (or video information) corresponding to the target object in the area where the target object is located, or can display the name of the target object in the area where the target object is located and display the text information (or video information) corresponding to the target object in an area whose distance from the area where the target object is located is less than a preset distance, or can also display the name of the target object in the area where the target object is located and display the text information (or video information) corresponding to the target object in a preset area.

[0097] When the associated information corresponding to the target object includes the text information and the video information corresponding to the target object, the viewing device can display the text information and the video information corresponding to the target object in the area where the target object is located, or can display the text information of the target object in the area where the target object is located and display the video information corresponding to the target object in an area whose distance from the area where the target object is located is less than a preset distance, or can also display the text information of the target object in the area where the target object is located and display the video information corresponding to the target object in a preset area.

[0098] When the associated information corresponding to the target object includes the name of the target object, as well as the text information and video information corresponding to the target object, the viewing device can display the name of the target object, as well as the text information and video information corresponding to the target object in the area where the target object is located. It can also display the name of the target object in the area where the target object is located, and display the text information and video information corresponding to the target object in an area whose distance from the area where the target object is located is less than a preset distance. Additionally, it can display the name of the target object in the area where the target object is located, and display the text information and video information corresponding to the target object in a preset area.

[0099] Exemplarily, when the user's eyes are gazing at a building, the viewing device can display the outline of the building. Subsequently, when the user presses the OK key or the trigger button, the viewing device can display the video file corresponding to the building. When the user selects the video file corresponding to the building and presses the OK key or the trigger button, the viewing device can play the video information.

[0100] Exemplarily, when the user's eyes are gazing at a mountain range, the viewing device can display the outline of the mountain range, as well as the text information and / or video information of the mountain range. The introduction panel of the text information and the position of the video information can be displayed at different locations.

[0101] Exemplarily, please refer to Figure 8 , Figure 8 which is a schematic diagram showing the display of the associated information of a mountain range disclosed in an embodiment of the present application. As Figure 8 shown, the introduction panel of the text information and the video information can be displayed above the cursor. The cursor is located in the area where the mountain range is located.

[0102] It should be understood that the above is an exemplary description of the display of the name of the target object, as well as the text information and video information corresponding to the target object, and does not limit the display method and display position of the name of the target object, as well as the text information and video information corresponding to the target object.

[0103] In some embodiments, the object can be an object of the city type or an object of the scenic spot type. Exemplarily, a building can be a building of the city type or a building of the scenic spot type. According to the importance level, the object can be a key object or a non-key object. The object can also be referred to as a Point Of Interest (POI).

[0104] Exemplarily, please refer to Figure 9 , Figure 9 which is a schematic diagram of a Point of Interest of the city type disclosed in an embodiment of the present application. As Figure 9As shown, the points of interest of the urban type can include key points of interest and non-key points of interest. The key points of interest of the urban type can include iconic scenic spots, iconic facilities, and iconic institutions. The non-key points of interest of the urban type can include tourist attractions, cultural venues, companies, historical events, ecology, healthcare, government agencies, entertainment, educational institutions, transportation facilities and services, etc.

[0105] Exemplarily, please refer to Figure 10 , Figure 10 which is a schematic diagram of a point of interest of the scenic spot type disclosed in an embodiment of the present application. As Figure 10 shown, the points of interest of the scenic spot type can include key points of interest and non-key points of interest. The key points of interest of the scenic spot type can include general views of the scenery, general views of the ecology, general views of the culture, and general views of the history. The non-key points of interest of the urban type can include mythical scenic spots, plant resources, food culture, "Ming and Qing Dynasties, Tang and Song Dynasties, Yuan and Ming Dynasties", check-in scenic spots, animal resources, ethnic culture, modern and contemporary, etc.

[0106] In some embodiments, the importance levels of the target objects are different, and the associated information corresponding to the target objects can be different.

[0107] When the target object is a key object, the viewing device can display the text information of the target object and / or the video information corresponding to the target object in response to the user's selection operation on the target object in the target map.

[0108] When the target object is a key object, the associated information corresponding to the target object can include the text information corresponding to the target object, or can also include the text information corresponding to the target object and the video information corresponding to the target object. Exemplarily, please refer to Figure 11 , Figure 11 which is a schematic diagram of the associated information corresponding to a key point of interest disclosed in an embodiment of the present application. Figure 11 The text introduction in

[0109] When the target object is a non-key object, the viewing device can display one or more of the name information of the target object, the text information corresponding to the target object, and the video information corresponding to the target object in response to the user's selection operation on the target object in the target map.

[0110] When the target object is a non-key object, the associated information corresponding to the target object can include the name of the target object, or can be the text information corresponding to the target object, or can also include the text information corresponding to the target object and the video information corresponding to the target object. Exemplarily, please refer to Figure 12 , Figure 12 which is a schematic diagram of the associated information corresponding to a non-key point of interest disclosed in an embodiment of the present application.

[0111] In some embodiments, a viewing device may include a mixed reality device. At this time, a viewing device can be used by a user to view a scene. The above is described with a viewing device including a mixed reality device.

[0112] In some embodiments, a viewing device may include multiple mixed reality devices. At this time, a viewing device can be used by multiple users to view a scene simultaneously, which can improve the utilization rate of the viewing device. When the viewing device includes multiple mixed reality devices, the viewing device may include a processing device. It can be seen that multiple mixed reality devices can share a processing device, which can reduce the complexity of the viewing device.

[0113] The user can interact with the viewing device through the mixed reality device.

[0114] Multiple mixed reality devices can be used independently, or synchronously, or some mixed reality devices can be used independently and some can be used synchronously.

[0115] The following takes a viewing device including two mixed reality devices, namely a first mixed reality device and a second mixed reality device, as an example for description.

[0116] When the first mixed reality device and the second mixed reality device are used synchronously, the viewing device can, in response to a user's selection operation on a target object in a target map, display the associated information corresponding to the target object through the first mixed reality device and the second mixed reality device respectively, that is, the first mixed reality device displays the associated information corresponding to the target object, and the second mixed reality device displays the associated information corresponding to the target object. The user here can be one of the two users using the first mixed reality device and the second mixed reality device.

[0117] Exemplarily, when two users want to use the viewing device and need to view the scene synchronously, the two users can use the first mixed reality device and the second mixed reality device respectively. During the viewing process of the two users, the two users can view the same object so that they can watch, discuss, introduce, etc. this object together. For example, when the two users are an adult and a child, the adult can introduce the viewed object to the child. For example, when the two users are adults, the two adults can discuss the viewed object with each other.

[0118] In one case, the usage modes of the two mixed reality devices included in the viewing device are fixed and are synchronous use.

[0119] In another case, the usage modes of the two mixed reality devices included in the viewing device are not fixed but can be set. The user can set them to synchronous use or independent use according to needs.

[0120] When the user needs to use two mixed reality devices of the viewing device synchronously, the user can set their usage mode to synchronous usage through the first mixed reality device or the second mixed reality device. After the two mixed reality devices are set to synchronous usage, the viewing device can adjust the positions of the first mixed reality device and / or the second mixed reality device so that the viewing angles of the first mixed reality device and the second mixed reality device are the same, so that the scenes viewed by the two users are the same.

[0121] Exemplarily, when a user sets through the first mixed reality device, after the viewing device detects the setting operation of synchronous usage by the user, it can adjust the position of the second mixed reality device so that the viewing angle of the second mixed reality device is the same as that of the first mixed reality device.

[0122] When the usage modes of the first mixed reality device and the second mixed reality device are synchronous usage, the viewing device can have two usage modes, namely the master-slave usage mode and the synchronous usage mode.

[0123] When the usage mode of the viewing device is the master-slave usage mode, one of the first mixed reality device and the second mixed reality device is the master mixed reality device, and the other mixed reality device is the slave mixed reality device. The master mixed reality device has a control effect on the slave mixed reality device.

[0124] When the usage mode of the viewing device is the synchronous usage mode, the first mixed reality device and the second mixed reality device have the same functions. The viewing device responds to the selection operation of the third target object in the target map by the first user, and displays the associated information corresponding to the third target object through the first mixed reality device and the third mixed display device. Similarly, the viewing device responds to the selection operation of the fourth target object in the target map by the second user, and displays the associated information corresponding to the fourth target object through the first mixed reality device and the third mixed display device. The first user is the user who uses the first mixed reality device. The second user is the user who uses the second mixed reality device.

[0125] When the first user views the third target object in the target map through the first mixed reality device, the second user can also synchronously view the third target object in the target map through the second mixed reality device; when the second user views the fourth target object in the target map through the second mixed reality device, the first user can also synchronously view the fourth target object in the target map through the first mixed display device.

[0126] It can be seen that when multiple mixed reality devices of the viewing device are used synchronously, the positions of the multiple mixed display devices remain consistent. Therefore, when the position of one mixed display device is changed by the user, the positions of the other mixed reality devices also need to change accordingly, so as to ensure that the content viewed by users using multiple mixed display devices remains synchronized.

[0127] When the user needs to use two mixed reality devices of the viewing device independently, the user can set their usage mode to independent use through the first mixed reality device or the second mixed reality device.

[0128] In some embodiments, the viewing device can collect the current pose information of the first user, can collect the current pose information of the second user, then can obtain first map data from the processing device according to the current pose information of the first user, can obtain second map data from the processing device according to the current pose information of the second user, can display a first target map through the first mixed reality device according to the first map data, can display a second target map through the second mixed reality device according to the second map data, can display the associated information corresponding to the first target object in the first target map through the first mixed reality device in response to the selection operation of the first target object in the first target map by the first user to obtain a third target map, and can display the associated information corresponding to the second target object in the second target map through the second mixed reality device in response to the selection operation of the second target object in the second target map by the second user to obtain a fourth target map.

[0129] It can be seen that when multiple mixed reality devices are used independently, the interactions between multiple mixed reality devices and different users are independent, and the interaction of one mixed reality device does not affect the interaction of another mixed display device, which can improve the user experience and the flexibility of interaction.

[0130] In some embodiments, the viewing device can display the third target map through the second mixed reality device in response to the sharing operation of the third target map by the first user.

[0131] In some embodiments, the viewing device can display the fourth target map through the first mixed reality device in response to the sharing operation of the fourth target map by the second user.

[0132] It can be seen that when one user views the scenery through one mixed reality device and wants to share the scenery content with another user, sharing can be performed through the sharing operation, which can improve the user experience and enrich the functions of the viewing device.

[0133] In some embodiments, the viewing device may display a small model corresponding to a target object in response to a user's operation of displaying the small model of the target object in the target map. The area where the target object is located may be the city where the target object is located, or the county where the target object is located, or the district where the target object is located, or other areas where the target object is located.

[0134] Exemplarily, when the viewing device detects the user's gaze at the target object through the visual tracking module and detects the user's operation on the trigger, the preset mechanical button or the preset function button, it may obtain the small model corresponding to the target object and display the small model corresponding to the target object. The small model corresponding to the target object may include a rectangular base, a 3D map model, the position information of the target object, etc.

[0135] Please refer to Figure 13 , Figure 13 which is a schematic flowchart of another interaction method disclosed in the embodiments of the present application. As Figure 13 shown, the interaction method may include the following steps.

[0136] 1301. The viewing device collects the current pose information of the user.

[0137] Among them, for the detailed description of step 1301, reference may be made to the description of step 201.

[0138] 1302. The viewing device sends the current pose information of the user to the processing device.

[0139] Correspondingly, the processing device receives the current pose information of the user from the viewing device.

[0140] 1303. The processing device determines the viewing angle information of the user according to the current pose information of the user, and determines the map data according to the viewing angle information of the user.

[0141] 1304. The processing device sends the map data to the viewing device.

[0142] Correspondingly, the viewing device receives the map data from the processing device.

[0143] Among them, for the detailed description of steps 1302 to 1304, reference may be made to the relevant description below step 202.

[0144] 1305. The viewing device displays the target map according to the obtained map data.

[0145] Among them, for the detailed description of step 1305, reference may be made to the description of step 203.

[0146] 1306. In response to the user's selection operation on the target object in the target map, display the associated information corresponding to the target object.

[0147] Among them, for the detailed description of step 1306, reference can be made to the description of step 204.

[0148] It should be understood that the same or corresponding information in different embodiments above can be referred to each other.

[0149] Please refer to Figure 14 , Figure 14 is a schematic structural diagram of an interaction device disclosed in an embodiment of the present application. Among them, the interaction device can be applied to a viewing device in a viewing system, and the viewing system can further include a processing device. As Figure 14 shown, the interaction device may include:

[0150] An acquisition unit 1401, configured to acquire the current pose information of the user;

[0151] An acquisition unit 1402, configured to acquire map data from the processing device according to the current pose information of the user;

[0152] A display unit 1403, configured to display a target map according to the map data;

[0153] The display unit 1403 is further configured to display associated information corresponding to the target object in response to a selection operation of the target object in the target map by the user.

[0154] In some embodiments, that the display unit 1403 displays the associated information corresponding to the target object in response to a selection operation of the target object in the target map by the user includes:

[0155] Highlight the target object when it is detected that the user is gazing at the target object in the target map;

[0156] Display the associated information corresponding to the target object in response to a confirmation operation of the target object by the user.

[0157] In some embodiments, that the display unit 1403 displays the associated information corresponding to the target object in response to a confirmation operation of the target object by the user includes:

[0158] When the gaze duration of the user on the target object is equal to a preset duration, display the associated information corresponding to the target object.

[0159] In some embodiments, the current pose information of the user includes the current position information and the current orientation information of the user, the viewing device includes a pose acquisition device, and the pose acquisition device includes a positioning module and a visual tracking module;

[0160] The acquisition unit 1401 is specifically configured to:

[0161] Acquire the current position information of the user through the positioning module;

[0162] Collect the current orientation information of the user through the visual tracking module.

[0163] In some embodiments, the viewing device includes a first mixed reality device and a second mixed reality device. The display unit 1403, in response to a user's selection operation on a target object in the target map, displays associated information corresponding to the target object, including:

[0164] In response to a user's selection operation on a target object in the target map, display the associated information corresponding to the first target object through the first mixed reality device and the second mixed reality device respectively.

[0165] In some embodiments, the viewing device includes a first mixed reality device and a second mixed reality device. The acquisition unit 1401 is specifically configured to:

[0166] Collect the current pose information of the first user, where the first user is the user using the first mixed reality device;

[0167] Collect the current pose information of the second user, where the second user is the user using the second mixed reality device;

[0168] The obtaining unit 1402 is specifically configured to:

[0169] Obtain first map data from the processing device according to the current pose information of the first user;

[0170] Obtain second map data from the processing device according to the current pose information of the second user;

[0171] The display unit 1403 displays the target map according to the map data, including:

[0172] Display the first target map through the first mixed reality device according to the first map data;

[0173] Display the second target map through the second mixed reality device according to the second map data;

[0174] The display unit 1403, in response to a user's selection operation on a target object in the target map, displays associated information corresponding to the target object, including:

[0175] In response to the first user's selection operation on the first target object in the first target map, display the associated information corresponding to the first target object through the first mixed reality device to obtain a third target map;

[0176] In response to the second user's selection operation on the second target object in the second target map, display the associated information corresponding to the second target object through the second mixed reality device.

[0177] In some embodiments, the display unit 1403 is further configured to display the third target map through the second mixed reality device in response to a sharing operation of the first user on the third target map.

[0178] In some embodiments, when the display unit 1403 responds to a selection operation of a user on a target object in a target map, the associated information corresponding to the target object displayed includes:

[0179] In the case where the target object is a key object, when responding to a selection operation of the user on the target object in the target map, display the text information of the target object and / or the video information corresponding to the target object;

[0180] In the case where the target object is a non-key object, when responding to a selection operation of the user on the target object in the target map, display one or more of the name information of the target object, the text information corresponding to the target object, and the video information corresponding to the target object.

[0181] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described acquisition unit 1401, acquisition unit 1402, and display unit 1403 can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0182] In several embodiments provided in the present application, the coupling between units can be electrical, mechanical, or other forms of coupling.

[0183] In addition, in each embodiment of the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0184] Please refer to Figure 15 , Figure 15 is a schematic structural diagram of a viewing device disclosed in an embodiment of the present application. As Figure 15 shown, the viewing device may include a processor 1501 and a memory 1502. The memory 1502 may store one or more computer programs. The one or more computer programs are configured to execute the method described in the foregoing method embodiments. The memory 1502 may exist independently or may be integrated with the processor 1501.

[0185] The processor 1501 may include one or more processing cores. The processor 1501 can be connected to various parts within the viewing device through various interfaces and circuits. It can execute various functions of the viewing device and process data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1502, and by calling the data stored in the memory 1502. Optionally, the processor 1501 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), or programmable logic array (PLA). The processor 1501 can integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 1501 and can be implemented separately through a communication chip.

[0186] The memory 1502 may include a random access memory (RAM) and may also include a read-only memory (ROM). The memory 1502 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1502 can include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above method embodiments, etc. The data storage area can also store data created during the use of the viewing device (such as phone book, audio and video data, chat record data, etc.).

[0187] When the computer program instructions stored in the memory 1502 are executed, the processor 1501 can be used to execute various operations performed by the viewing device in the above method embodiments. The specific implementation of these operations can be referred to the previous embodiments and will not be elaborated here.

[0188] A schematic structural diagram of a computer-readable storage medium disclosed in an embodiment of the present application. Computer program code is stored in the computer-readable medium, and the computer program code can be called by the processor to execute various operations in the above method embodiments. The specific implementation of each of the above operations can be referred to the previous embodiments and will not be elaborated here.

[0189] A computer-readable storage medium may be an electronic memory such as a flash memory, an electrically erasable programmable read only memory (EEPROM), an erasable programmable read only memory (EPROM), a hard disk, or a ROM. Optionally, the computer-readable storage medium may include a non-transitory computer-readable storage medium. The computer-readable storage medium has a storage space for program codes for performing any of the method steps in the above-described methods. These computer program codes may be read from or written into one or more computer program products. The computer program codes may be compressed in a suitable form, for example.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An interactive method, characterized in that: The method is applied to a viewing device in a viewing system, and the viewing system further includes a processing device, including: Collect the user's current posture information; Acquiring map data from the processing device according to the current posture information; Displaying a target map according to the map data; In response to a user's selection operation on a target object in the target map, associated information corresponding to the target object is displayed.

2. The method according to claim 1, characterized in that In response to the user's selection operation on the target object in the target map, displaying the associated information corresponding to the target object includes: When detecting that the user is gazing at a target object in the target map, highlighting the target object; In response to a user's confirmation operation on the target object, associated information corresponding to the target object is displayed.

3. The method according to claim 2, characterized in that In response to the user's confirmation operation on the target object, displaying the associated information corresponding to the target object includes: When the user's gaze duration on the target object is equal to a preset duration, the associated information corresponding to the target object is displayed.

4. The method according to claim 1, characterized in that: The current posture information of the user includes the current position information and current orientation information of the user, the viewing device includes a posture acquisition device, and the posture acquisition device includes a positioning module and a visual tracking module; The collecting of the user's current posture information includes: Collecting the current location information of the user through the positioning module; The current orientation information of the user is collected through the visual tracking module.

5. The method according to claim 1, characterized in that The viewing device includes a first mixed reality device and a second mixed reality device, and the displaying of the associated information corresponding to the target object in the target map in response to a user's selection operation on the target object includes: In response to a user's selection operation on a target object in the target map, associated information corresponding to the first target object is displayed respectively through the first mixed reality device and the second mixed reality device.

6. The method according to claim 1, characterized in that The viewing device includes a first mixed reality device and a second mixed reality device, and the collecting of the current posture information of the user includes: Collecting current posture information of a first user, where the first user is a user using the first mixed reality device; Collecting current posture information of a second user, where the second user is a user using the second mixed reality device; The acquiring map data from the processing device according to the current posture information comprises: Acquiring first map data from the processing device according to the current position information of the first user; acquiring second map data from the processing device according to the current position information of the second user; The displaying of the target map according to the map data comprises: Displaying a first target map through the first mixed reality device according to the first map data; Displaying a second target map through the second mixed reality device according to the second map data; In response to the user's selection operation on the target object in the target map, displaying the associated information corresponding to the target object includes: In response to the first user's selection operation on the first target object in the first target map, displaying the associated information corresponding to the first target object through the first mixed reality device to obtain a third target map; In response to the second user's selection operation on the second target object in the second target map, the associated information corresponding to the second target object is displayed through the second mixed reality device.

7. The method according to claim 6, characterized in that The method further comprises: In response to the first user's sharing operation on the third target map, the third target map is displayed through the second mixed reality device.

8. The method according to any one of claims 1 to 6, characterized in that: In response to the user's selection operation on the target object in the target map, displaying the associated information corresponding to the target object includes: In the case where the target object is a key object, in response to a user's selection operation of the target object in the target map, displaying text information of the target object and / or video information corresponding to the target object; In the case where the target object is a non-key object, in response to a user's selection operation of the target object in the target map, one or more of the name information of the target object, the text information corresponding to the target object, and the video information corresponding to the target object are displayed.

9. An interactive device, characterized in that: The device is applied to a viewing device in a viewing system, and the viewing system further includes a processing device, including: A collection unit, used to collect the user's current posture information; an acquisition unit, configured to acquire map data from the processing device according to the current posture information; a display unit, for displaying a target map according to the map data; The display unit is further used to display the associated information corresponding to the target object in response to the user's selection operation on the target object in the target map.

10. A viewing device, characterized in that: The method comprises a processor and a memory, wherein the processor is used to call a computer program stored in the memory to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or a computer instruction. When the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images