Virtual content display method and system, electronic equipment and storage medium

Abstract

The invention discloses a virtual content display method and system, electronic equipment and a storage medium. The virtual content display method is applied to the display equipment. The method comprises the following steps: acquiring a virtual scene model corresponding to a scene to be viewed, wherein the scene to be viewed comprises a plurality of scene objects; marking first attribute information of the plurality of scene objects on the virtual scene model; obtaining a target scene object from the plurality of scene objects; and displaying a virtual scene image of the target scene object and second attribute information in a specified display mode, wherein the second attribute information and the first attribute information are different in type and display form. According to the method, the scene object to be viewed by the user is acquired as the target scene object, and the virtual scene image and the second attribute information of the target scene object are displayed at the same time in the specified display mode, so that the viewing of the target scene object becomes real and vivid, and the viewing experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of display technologies, and more particularly, to a method, a system, an electronic device, and a storage medium for displaying virtual content. Background Art

[0002] Mixed Reality (MR) refers to the technology of superimposing real objects into a virtual world, in contrast to Augmented Reality (AR) which superimposes virtual objects onto the real world. With the rapid development of intelligent terminals and MR technologies in recent years, more and more places are equipped with electronic viewing systems to help users quickly understand relevant information about the content being viewed. However, most current electronic viewing systems are fixedly configured, and the presentation forms of the content being viewed are monotonous and rigid, greatly reducing the viewing experience of users. Summary of the Invention

[0003] The present application provides a method, a system, an electronic device, and a storage medium for displaying virtual content, so as to improve the problem that the viewing form of virtual content is too monotonous and rigid to meet the personalized viewing needs of users.

[0004] In a first aspect, an embodiment of the present application provides a method for displaying virtual content, which is applied to a display device. The method includes: obtaining a virtual scene model corresponding to a scene to be viewed, where the scene to be viewed includes a plurality of scene objects; marking first attribute information of the plurality of scene objects on the virtual scene model; obtaining a target scene object from the plurality of scene objects; and displaying a virtual scene image of the target scene object and second attribute information in a specified display manner, where the type and presentation form of the second attribute information are different from those of the first attribute information.

[0005] In a second aspect, an embodiment of the present application provides a system for displaying virtual content. The display system includes a display device and a terminal device, where the display device is communicatively connected to the terminal device. The terminal device is configured to obtain a virtual scene model corresponding to a scene to be viewed, where the scene to be viewed includes a plurality of scene objects; and mark first attribute information of the plurality of scene objects on the virtual scene model. The display device is configured to obtain a target scene object from the plurality of scene objects; and display a virtual scene image of the target scene object and second attribute information in a specified display manner, where the type and presentation form of the second attribute information are different from those of the first attribute information.

[0006] In a third aspect, the present application provides an electronic device, including one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method of the first aspect above.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium storing program code, where the method of the first aspect is executed when the program code runs.

[0008] A method, system, electronic device, and storage medium for displaying virtual content provided by the present application. The method includes obtaining a virtual scene model corresponding to a scene to be viewed, where the scene to be viewed includes multiple scene objects; marking first attribute information of the multiple scene objects on the virtual scene model; obtaining a target scene object from the multiple scene objects; and displaying a virtual scene image of the target scene object and second attribute information in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information. In this way, the scene object that the user wants to view can be obtained as the target scene object, and the virtual scene image of the target scene object and the second attribute information are simultaneously displayed in a specified display manner, making the viewing of the target scene object real and vivid and improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application, and those skilled in the art can obtain other drawings without creative efforts based on these drawings.

[0010] Figure 1 Shows a flowchart of a method for displaying virtual content provided by an embodiment of the present application.

[0011] Figure 2 Shows an example diagram of POI icons corresponding to different categories of scene objects provided by an embodiment of the present application.

[0012] Figure 3 Shows another example diagram of POI icons corresponding to different categories of scene objects provided by an embodiment of the present application.

[0013] Figure 4 Shows a virtual scene model marked with first attribute information of multiple scene objects provided by an embodiment of the present application.

[0014] Figure 5 Shows an example diagram of the display of the search results of the target scene object in the prior art.

[0015] Figure 6The figure shows an example diagram of a virtual scene image and second attribute information of a target scene object displayed in a specified display manner provided by an embodiment of the present application.

[0016] Figure 7 The figure shows an example diagram in which the second attribute information provided by an embodiment of the present application changes with different target scene objects.

[0017] Figure 8 The figure shows another example diagram in which the second attribute information provided by an embodiment of the present application changes with different target scene objects.

[0018] Figure 9 The figure shows a flowchart of a method for displaying virtual content provided by another embodiment of the present application.

[0019] Figure 10 The figure shows a flowchart of a method for displaying virtual content provided by still another embodiment of the present application.

[0020] Figure 11 The figure shows an example diagram of an observation glasses provided by an embodiment of the present application.

[0021] Figure 12 The figure shows an example effect diagram of viewing a virtual image and a real scene image of a scene object through the gameplay of the observation glasses provided by an embodiment of the present application.

[0022] Figure 13 The figure shows a flowchart of a method for displaying virtual content provided by still another embodiment of the present application.

[0023] Figure 14 The figure shows an example diagram of displaying ranging information between different scene objects provided by an embodiment of the present application.

[0024] Figure 15 The figure shows a structural block diagram of a virtual content display system provided by an embodiment of the present application.

[0025] Figure 16 The figure shows a structural block diagram of an electronic device provided by an embodiment of the present application.

[0026] Figure 17 The figure shows a storage unit for storing or carrying program codes for implementing the method for displaying virtual content according to an embodiment of the present application. Detailed implementation manners

[0027] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0028] In an embodiment of the present application, the inventors propose a method, a system, an electronic device, and a storage medium for displaying virtual content. By obtaining a virtual scene model corresponding to a scene to be viewed, where the scene to be viewed includes multiple scene objects; marking first attribute information of the multiple scene objects on the virtual scene model; obtaining a target scene object from the multiple scene objects; and displaying a virtual scene image and second attribute information of the target scene object in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information. Thus, through the above method, the scene object that the user wants to view can be obtained as the target scene object, and the virtual scene image and the second attribute information of the target scene object can be displayed simultaneously in a specified display manner, making the viewing of the target scene object real and vivid and improving the user's viewing experience.

[0029] The following will specifically describe each embodiment of the present application with reference to the accompanying drawings.

[0030] Please refer to Figure 1 , which shows a flowchart of a method for displaying virtual content provided by an embodiment of the present application. This embodiment provides a method for displaying virtual content, which can be applied to a display device. The display device can be a device with the function of displaying a virtual-real fusion image. The display device can be integrated or split. Optionally, if the display device is integrated, a lens with the function of displaying a virtual-real fusion image is integrated on the display device, and the user can directly view the virtual-real fusion image through the display device; if the display device is split, the display device can include a head-mounted display device and a terminal device that communicates with the head-mounted display device. At this time, the head-mounted display device can be a head-mounted display device such as a VR glasses or a helmet, and the specific type of the display device can be not limited. The method includes:

[0031] Step S110: Obtain a virtual scene model corresponding to a scene to be viewed, where the scene to be viewed includes multiple scene objects.

[0032] In the embodiment of the present application, the scene to be viewed can be any scene that the user wants to view, that is, the specific scene type of the scene to be viewed can be not limited. Exemplarily, the scene to be viewed can be a tourism scene, a traffic scene, an urban scene, etc.

[0033] The scene to be viewed may include an area or a range of areas, and each scene to be viewed may include multiple scene objects. For example, taking the scene to be viewed as a tourist scene as an example, assuming the scenic spot of this tourist scene is "Huashan Mountain", then the scene to be viewed includes the area or range of areas enclosed by the scenic spot "Huashan Mountain". Each mountain in the scenic spot "Huashan Mountain" can be regarded as a scene object of the scene to be viewed. Or, taking the scene to be viewed as an urban scene as an example, assuming there are multiple buildings in this urban scene, then each building can be regarded as a scene object.

[0034] The scene to be viewed is a real scene, and the virtual scene model corresponding to the scene to be viewed is the virtual image corresponding to the aforementioned real scene. Optionally, the virtual image can be a two-dimensional or multi-dimensional (for example, it can be three-dimensional, four-dimensional, etc., and the specific number of dimensions can be not limited) map. In the embodiments of the present application, the virtual scene model is taken as a three-dimensional model for illustration.

[0035] In order to facilitate more convenient knowledge of the scene object that the user wants to view subsequently, the virtual scene model corresponding to the scene to be viewed can be obtained, that is, a three-dimensional virtual scene model identical to the real scene content is constructed. Optionally, the scene to be viewed can be an indoor scene or an outdoor scene.

[0036] As a way, the information of the scene to be viewed can be obtained first; then, based on the information of the scene to be viewed, the virtual scene model corresponding to the scene to be viewed is obtained.

[0037] As a specific implementation manner, the corresponding virtual scene model can be obtained by means of the BIM (Building Information Modeling) data or 3D model information of the scene to be viewed, or the corresponding virtual scene model can be obtained by means of the GIS (Geographic Information System) data or map navigation data of the scene to be viewed.

[0038] It should be noted that in the embodiments of the present application, the content of the scene to be viewed can be edited (such as adding or modifying, etc.), and the edited content can include information such as name, geographical coordinates, height, pitch value, yaw value, roll value, etc. Exemplarily, taking the scene to be viewed as a tourist scene as an example, multiple different viewpoints can be created (each viewpoint can be understood as a scene object), and the longitudes, latitudes, and heights of different viewpoints are different. Attribute settings can be added to each newly created viewpoint, and the attribute settings can include content such as viewpoint name, center point latitude, center point longitude, height, category, and associated multimedia materials.

[0039] Optionally, in some other embodiments, a specified number of scene objects may also be selected as the scene to be viewed according to user needs, and the specific value of the specified number may not be limited. For example, taking the above-mentioned tourism scene as an example, the scenic spot "Huashan" contains many peaks. The user can only select the East Peak, West Peak, South Peak, and North Peak of "Huashan" as the scene to be viewed. Similarly, for the case where the tourism location is a city, only several landmark buildings can be selected as the scene to be viewed, or all the buildings in the city can be selected as the scene to be viewed (that is, at this time, scene objects can be selected according to the preset information of the scene information. For example, if a city has 2,000 buildings pre-edited, then these 2,000 buildings can be used as the scene to be viewed), that is, the user can adjust the specific content of the scene to be viewed according to needs.

[0040] Step S120: Mark the first attribute information of the multiple scene objects on the virtual scene model.

[0041] In the case of obtaining the virtual scene model corresponding to the scene to be viewed, for the convenience of subsequent data processing, the virtual scene model can be aligned with the real scene to be viewed at the data level, that is, the data of the scene to be viewed is added to the corresponding virtual scene model in the form of attribute addition.

[0042] As an implementation manner, the first attribute information of multiple scene objects can be marked on the virtual scene model, that is, the first attribute information of each scene object is marked to initially help the user quickly and intuitively distinguish different scene objects. Among them, the first attribute information may include information such as the name and category of the scene object. Among them, the name can be displayed in text form, and the category can be displayed in the form of a POI icon. POI (Point of Information) icons corresponding to different categories of scene objects are different.

[0043] In a specific application scenario, please refer to Figure 2 , which shows an example diagram of POI icons corresponding to different categories of scene objects provided by the embodiments of the present application. Taking the scene to be viewed as an urban scene as an example, as Figure 2As shown in the figure, the urban scene includes category scenes such as entertainment attractions, public facilities, government, and enterprises. Each category scene can include multiple scene objects, and the POI icons of each scene object are unique (here, "unique" can be understood as the POI icons of different scene objects are different from each other, but the specific display style of the POI icon of each scene object can be adjusted according to actual needs. For example, the shape of the POI icon can be modified). Exemplarily, the entertainment attractions include scene objects such as "landmark attractions, tourist attractions, entertainment and leisure, ecology, and historical events", the public facilities include scene objects such as "landmark facilities, transportation facility services, cultural venues, healthcare, and educational institutions", and the government and enterprises include scene objects such as "landmark institutions, government agencies, and corporate enterprises". The POI icons of different scene objects are different. Optionally, the entertainment attractions can also include intelligent recommendations (such as robots, voice assistants), parent-child parks, drinks and snacks, chess and card rooms, food, etc. scene objects ( Figure 2 not shown). Similarly, the urban scene includes scene objects such as "hospitals, schools, parks, government, residences", and the POI icons of these scene objects are different.

[0044] For some to-be-viewed scenes of pure attractions, they can include scene objects such as "scenery overview, ecological overview, cultural overview, historical overview, mythological attractions, plant resources, food culture, modern times, check-in attractions, animal resources, ethnic culture, Ming and Qing dynasties, Tang and Song dynasties, Yuan and Ming dynasties", and the POI icons of some scene objects are as Figure 3 shown. Similarly, the to-be-viewed scenes of pure attractions can also include intelligent recommendations (such as robots, voice assistants), AI experiences, character DIY, hotel and homestays, student areas, fairy tale parks, etc. scene objects ( Figure 3 not shown).

[0045] In a specific application scenario, please refer to Figure 4 , which shows a virtual scene model marked with the first attribute information of multiple scene objects provided by an embodiment of the present application. At this time, the virtual scene model is a two-dimensional map, and information such as the names of different buildings and POI icons can be displayed on the two-dimensional map. For example, Figure 4 the two-dimensional map shown displays the names and POI icons of scene objects such as "Renju Tianfu Shijia, Golden Apple New Montessori Kindergarten, Bank of China, Paotongshu Primary School, Jincheng Avenue". Optionally, the first attribute information can be understood as relatively fixed attribute information, that is, the attribute information that each scene object will display.

[0046] Step S130: Obtain a target scene object from the multiple scene objects.

[0047] In the embodiments of the present application, the target scene object represents a scene object within the field of view angle of the user. The number of target scene objects can be one or more, and the specific number can be not limited. For example, there are thousands of trees in a forest, but only a dozen trees are seen from the current perspective of the user. Then these dozen trees can be used as the target scene objects.

[0048] As an implementation, at least one scene object can be selected from multiple scene objects as the target scene object.

[0049] Step S140: Display the virtual scene image of the target scene object and the second attribute information in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information.

[0050] Among them, the type of the second attribute information is different from that of the first attribute information. For example, the second attribute information can include the outline of the target scene object, or can include the multimedia introduction information of the target scene object and the outline of the target scene object. The display form of the second attribute information is also different from that of the first attribute information. For example, the first attribute information can be displayed in the form of text and / or POI icons; while the multimedia introduction information in the second attribute information can be displayed in the form of pictures, videos or a combination of pictures and videos, or the second attribute information can also be displayed in a way that combines text, pictures and videos. The outline of the target scene object can be displayed in the way of "highlighted line drawing". It should be noted that "highlighted line drawing" is just an example of a display style, and the display style of the outline of the target scene object can be adjusted according to needs in actual implementation.

[0051] In this embodiment, a virtual display device can be used to display a virtual scene image of a target scene object and second attribute information. Then, as an implementation, to facilitate the user's awareness of the scene object for interaction, the virtual scene image of the target scene object and the second attribute information of the target scene object can be displayed by the virtual display device in a specified display manner. Among them, the specific content of the specified display manner can be set according to actual needs. For example, when displaying the second attribute information of the target scene object in the specified display manner, it can include highlighting the multimedia introduction information of the target scene object and highlighting the outline of the target scene object (as an implementation, the multimedia introduction information of the target scene object can be highlighted while highlighting the outline of the target scene object, that is, playing the multimedia introduction information of the target scene object). It should be noted that the highlighting here refers to highlighting the outline of the virtual scene image of the target scene object, and the highlighting can be understood as associating and displaying picture explanations, video introductions, etc. of the target scene object around the target scene object. The specific color of the highlighting can be adjusted according to actual needs. For example, it can be white highlighting, yellow highlighting, etc.

[0052] Optionally, in the embodiment of the present application, after highlighting the outline of the target scene object, the multimedia introduction information of the target scene object can be displayed. And, when there are multiple pieces of multimedia introduction information, they can be displayed in order of priority from high to low. For example, if the multimedia information of the target scene object includes text + picture + video, assuming the priority order of this multimedia information is video > picture > text, then the video can be displayed first, then the picture, and then the text. Optionally, the specific display form of the multimedia introduction information of the target scene object is only an example here and does not constitute a limitation to this solution.

[0053] In a specific application scenario, please refer to Figure 6 which shows an example diagram of displaying the virtual scene image of the target scene object and the second attribute information in the specified display manner provided by the embodiment of the present application. As Figure 6 shown, for a determined target scene object, the outline of the target object, that is, the outer circle of the building, can be highlighted. Figure 6 In the outer circle (outer perimeter) of the building is highlighted, and the outer circle of the picture of the location information of the building is highlighted (that is, adding a border to highlight it). Compared with the search result display manner shown in the prior art such as Figure 5 the display manner of this embodiment is more intuitive, clear, and understandable.

[0054] The types of the second attribute information in the embodiments of the present application may vary with different target scene objects, that is, the number of the second attribute information displayed corresponding to different target scene objects may vary, so as to improve the flexibility of virtual content display. For example, when the target scene object is a scenic spot, the second attribute information of the target scene object may vary with data such as the popularity of the scenic spot. For example, the higher the popularity of the scenic spot, the more types of the second attribute information it may include. Exemplarily, when the popularity of the scenic spot is the highest, the corresponding second attribute information may include pictures + text + videos + sound effects (the sound effects here may include special effects such as "fireworks" and "explosions"). When the popularity of the scenic spot relatively decreases, the corresponding second attribute information may include pictures + text + sound effects. When the popularity of the scenic spot is even lower, the corresponding second attribute information may include pictures + text.

[0055] For example, in a specific application scenario, please refer to Figure 7 and Figure 8 , both of which show example diagrams of the types or quantities of the second attribute information provided in the embodiments of the present application varying with different target scene objects. Still taking the tourism scene as an example, as Figure 7 shown, when the popularity of a certain scenic spot is relatively high, the second attribute information of this scenic spot may include text + pictures. Figure 8 The scenic spot shown in Figure 7 is different from the scenic spot shown in Figure 8 , and Figure 7 the popularity of the scenic spot shown in Figure 8 is higher than that of the scenic spot shown in

[0056] At this time, the second attribute information of the scenic spot shown in

[0057] By displaying the relevant information of the target scene object that the user wants to view in a specified display manner, it can clearly help the user view the interactive scene object and provide an immersive viewing experience.

[0058] The virtual content display method provided in this embodiment obtains a virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects; marks first attribute information of the multiple scene objects on the virtual scene model; obtains a target scene object from the multiple scene objects; and displays a virtual scene image of the target scene object and second attribute information in a specified display manner, where the types and display forms of the second attribute information are different from those of the first attribute information. Thus, in the above manner, the scene object that the user wants to view can be obtained as the target scene object, and the virtual scene image of the target scene object and the second attribute information are simultaneously displayed in a specified display manner, making the viewing of the target scene object real and vivid and improving the user's viewing experience.

[0059] Please refer to Figure 9 , which shows a flowchart of a virtual content display method provided in another embodiment of the present application. This embodiment provides a virtual content display method that can be applied to a display device. The method includes:

[0060] Step S210: Obtain a virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects.

[0061] Among them, the specific implementation of step S210 can refer to the relevant description in step S110 in the foregoing embodiment, and will not be elaborated here.

[0062] Step S220: Mark first attribute information of the multiple scene objects on the virtual scene model.

[0063] Among them, the specific implementation of step S220 can refer to the relevant description in step S120 in the foregoing embodiment, and will not be elaborated here.

[0064] Step S230: Obtain the pose information collected by the visual tracking module.

[0065] In this embodiment, the pose information of the user can be obtained by means of the display device, and then the target scene object can be determined according to the pose information.

[0066] Specifically, the display device may include a visual tracking module and a display module. The visual tracking module can determine the scene object that the user wants to interact with. As an implementation manner, the pose information collected by the visual tracking module of the display device can be obtained. Specifically, the current orientation of the user can be obtained according to the built-in sensors such as the geomagnetic sensor, gyroscope (or gyroscope combined with binocular cameras) and / or acceleration sensor of the display device, and then the heading angle of the display device in the Earth coordinate system (ENU) can be calculated through the sensed data, so as to further calculate the current rotation azimuth of the device, and then the pose information of the user can be obtained.

[0067] Among them, to obtain accurate current pose information of the user, taking the MR viewing platform at a fixed position as an example, since the position is fixed, real-time rotation information needs to be obtained, including orientation information in three directions: yaw, pitch, and roll. Specifically, an IMU sensor (including an accelerometer and a gyroscope, and optionally including a magnetometer) can be integrated inside the MR viewing platform hardware. Based on the fused data of the IMU sensor (such as data including the accelerometer and gyroscope), high-frame-rate orientation information in each direction such as yaw, pitch, and roll can be calculated. However, the rotation information solved based on the IMU generally has a drift problem, especially in the yaw direction, which is more obvious. To correct this drift problem, an infrared camera sensor can be additionally installed in the display device to capture infrared beacons fixed relative to the scene to be viewed, so as to obtain the absolute rotation information of the display device relative to the scene to be viewed. By fusing this information with the rotation information solved by the IMU, high-frame-rate drift-free rotation information can be obtained.

[0068] Optionally, the pose information may include at least one of head movement information, eye movement information, gesture information, and touch operation.

[0069] During the process of collecting pose information through the visual tracking module, there can be various interaction modes between the user and the display device. For example, it can include a head aiming mode, an eye movement interaction mode, a gesture interaction mode, and a manual interaction mode, etc. Among them, the pose information corresponding to the head aiming mode is head movement information, the pose information corresponding to the eye movement interaction mode is eye movement information, the pose information corresponding to the gesture interaction mode is gesture information, and the pose information corresponding to the manual interaction mode is touch operation.

[0070] For the head aiming interaction mode, a head motion tracker needs to be set in the display module of the display device as part of the input module, that is, the user's head movement is used as non-contact input information. The head orientation data obtained by sensors (such as angular motion sensors like gyroscopes) is compared with the preset information of each scenic spot (such as three-dimensional spatial positions). When the similarity between the two reaches a specified threshold (the specific value of the specified threshold can be not limited, for example, it can be 80%), it is determined that the user selects this scenic spot, and the preset information of the selected scenic spot / scenery, that is, content such as 3D contours, 4D special effects, animations, types, etc., is aligned and displayed on the display device side. Optionally, corresponding audio / video / text and other content can also be played.

[0071] For the eye movement interaction mode, an eye movement acquisition module needs to be set as part of the input module in the display module of the display device. The eye movement acquisition module is used to obtain the user's eye movement information (such as, line of sight trajectory, fixation point, etc.) as input information, and compare the head orientation data obtained by sensors (such as angular motion sensors like gyroscopes) with the preset information of each scenic spot (such as three-dimensional spatial position). When the similarity between the two reaches a certain threshold (similarly, the specific value of this threshold can also be not limited), it is determined that the user selects this scenic spot, and the preset information of the selected scenic spot / scenery is aligned and displayed at the head-mounted display end, that is, content such as 3D contour, 4D special effect, animation, type, etc. Optionally, corresponding audio / video / text and other content can also be played.

[0072] For the gesture interaction mode, interaction can be performed through the user's gestures, and the preset information of the selected scenic spot / scenery is aligned and displayed at the display device end, that is, content such as 3D contour, 4D special effect, animation, type, etc. Optionally, corresponding audio / video / text and other content can also be played.

[0073] For the manual interaction mode, a triggering device (such as a trigger) can be triggered as part of the input module. For example, it can be triggered by pulling the trigger to determine the scenic spot selected by the user.

[0074] As an implementation manner, the user can interact with the display device based on at least one of the above interaction modes, and then obtain the pose information during the interaction process through the visual tracking module of the display device. That is, the interaction modes can be used alternately or comprehensively, which improves the flexibility of obtaining pose information, and further improves the flexibility and intelligence of determining the target scene object.

[0075] Step S240: Obtain the target scene object from the multiple scene objects based on the pose information.

[0076] In the case where the pose information of the user's interaction process with the display device is obtained, the target scene object can be obtained from the multiple scene objects based on this pose information.

[0077] As an implementation manner, the corresponding viewing angle range can be obtained based on this pose information; then, at least one scene object among the multiple scene objects that falls within the viewing angle range is used as the target scene object. That is, the total number of scene objects is greater than or equal to the number of target scene objects, and the viewing angle range can be understood as the field of view angle range under the user's current pose. It should be noted that when the total number of scene objects is equal to the number of target scene objects, the number of scene objects includes at least two. By specifically determining the target scene object according to the viewing angle range, the accuracy of determining the target scene object is improved.

[0078] Among them, obtaining a corresponding viewing range based on the pose information may specifically include: obtaining a corresponding viewing range based on the user's head movement information; obtaining a corresponding viewing range based on the user's eye movement information; obtaining a corresponding viewing range based on the user's gesture information; and / or obtaining a corresponding viewing range based on the user's touch operation.

[0079] Step S250: Display the virtual scene image of the target scene object and the second attribute information in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information.

[0080] Among them, the specific implementation of step S250 may refer to the relevant description in step S140 in the foregoing embodiment, and will not be elaborated here.

[0081] The virtual content display method provided in this embodiment obtains a virtual scene model corresponding to the scene to be viewed by acquiring the virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects; marking the first attribute information of the multiple scene objects on the virtual scene model; acquiring the pose information collected by the visual tracking module; obtaining a target scene object from the multiple scene objects based on the pose information; and displaying the virtual scene image of the target scene object and the second attribute information in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information. Thus, through the above method, the scene object that the user wants to view can be obtained as the target scene object, and the virtual scene image of the target scene object and the second attribute information are simultaneously displayed in a specified display manner, making the viewing of the target scene object real and vivid and improving the user's viewing experience.

[0082] At the same time, by obtaining a corresponding viewing range based on the pose information; then, at least one scene object that falls within the viewing range among the multiple scene objects is used as the target scene object, enabling the user to participate in the determination process of the target scene object in an interactive manner and improving the interactive experience during the viewing of virtual content; at the same time, the user can complete the acquisition of pose information without any manual input, which can improve the convenience and intelligence of determining the target scene object, that is, making the determination of the target scene object more user-friendly.

[0083] Please refer to Figure 10 , which shows a flowchart of a virtual content display method provided in another embodiment of the present application. This embodiment provides a virtual content display method that can be applied to a display device. The method includes:

[0084] Step S310: Obtain a virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects.

[0085] Among them, the specific implementation of step S310 can refer to the relevant description in step S110 in the foregoing embodiments, and will not be elaborated here.

[0086] Step S320: Mark the first attribute information of the multiple scene objects on the virtual scene model.

[0087] Among them, the specific implementation of step S320 can refer to the relevant description in step S120 in the foregoing embodiments, and will not be elaborated here.

[0088] Step S330: Obtain the scene object keywords collected by the voice module.

[0089] In this embodiment, the display device may further include a voice module, and the voice module may specifically be some application programs or models with voice recognition and processing functions. Exemplarily, the voice module may be large language models such as ChatGPT and AndesGPT, and the specific type may not be limited.

[0090] As an implementation manner, the target scene object can be determined with the help of the voice module. Specifically, the scene object keywords collected by the voice module can be obtained first, that is, the voice content of the user is subjected to voice recognition to obtain the scene object keywords.

[0091] Step S340: Obtain the target scene object from the multiple scene objects based on the scene object keywords.

[0092] After the scene object keywords are recognized, the target scene object can be obtained from the multiple scene objects based on the scene object keywords. Specifically, the obtained scene keywords can be compared with the keywords of each of the multiple scene objects respectively, and then the scene object corresponding to the keyword with the same content as the obtained scene object keyword can be obtained as the target scene object, or the scene object corresponding to the keyword with a similarity higher than (greater than) a preset threshold (the specific value may not be limited, for example, it may be 95%) to the content of the obtained scene object keyword can be obtained as the target scene object.

[0093] Optionally, the number of target scene objects can be one or more, and the specific number may not be limited.

[0094] In a specific application scenario, assuming that the user's voice content is "What is the Shenzhen Agricultural Building?", the scene object keyword "Shenzhen Agricultural Building" can be recognized through the voice module. This scene object keyword is the name of the scene object. By comparing this scene object keyword with the names of the multiple scene objects respectively, it can be concluded that the "Shenzhen Agricultural Building" is the final target scene object.

[0095] It should be noted that for some voice content, there may be no obvious scene object keywords. In this case, the scene object closest to the user among the scene objects matching the voice content can be used as the final target scene object. At this time, the number of scene objects matching the voice content can be multiple. Optionally, in some embodiments, multiple scene objects matching the voice content can also be used as target scene objects.

[0096] Exemplarily, assume that the user's voice content is "Is there a place with convenient transportation nearby? I need to go there to take a taxi". At this time, the voice module can determine which scene object meets this condition from the current user's perspective (specifically, infrared light can be emitted towards the user's eyes, and the head orientation data obtained by sensors (such as angular motion sensors like gyroscopes) is compared with the preset information of each scene (such as three-dimensional spatial position) to determine the scene object from the current user's perspective). If there are multiple scene objects that meet the criteria, the preset information of one or more scene objects closest to the user can be displayed according to the user's current pose information (here, the preset information can be understood as the aforementioned first attribute information, second attribute information, and virtual scene models, etc.).

[0097] By determining the target scene object with the help of the user's voice content, on the one hand, it can make the determination of the target scene object more user-friendly and intelligent; on the other hand, there is no need to pull the trigger or perform other actions to interact with virtual content through voice, making the process of using the display device by the user as easy as chatting with people, bringing a good interaction experience.

[0098] Step S350: Display the virtual scene image of the target scene object and the second attribute information in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information.

[0099] Among them, the specific implementation of step S350 can refer to the relevant description in step S140 of the foregoing embodiments and will not be elaborated here.

[0100] In the embodiments of the present application, the virtual image and real scene image of the scene object can also be viewed through an observation glasses.

[0101] Please refer to Figure 11 , which shows an example diagram of an observation glasses provided by an embodiment of the present application. As Figure 11As shown, as an implementation, if it is detected that a certain scene object is magnified and viewed through the observation glasses, the real image of the viewed scene object can be displayed on the screen of the observation glasses while the virtual image of the scene object is displayed (the observation glasses can pre-obtain and store the virtual images of all scene objects from the display device), so that the virtual image and the real image of the scene object can be viewed simultaneously through the observation glasses.

[0102] In a specific application scenario, please refer to Figure 12 , which shows an example effect diagram of viewing the virtual image and the real image of a scene object through the gameplay of the observation glasses provided by the embodiment of the present application. As Figure 12 shown on the left of the figure, the current scene to be viewed can be aimed at through the observation glasses first, and then by continuously magnifying the actual scene of the scene to be viewed, the observation glasses are magnified and focused on a certain scene object that the user wants to view. At this time, the Figure 12 scene object shown on the right figure is obtained. Through the observation glasses, the user can not only see the real scene of the scene object, but also see the virtual image of the scene object.

[0103] Optionally, in the virtual image and the real image seen by the user through the observation glasses, as Figure 12 shown on the right figure, the outline of the current building (i.e., the scene object) is highlighted to allow the user to intuitively distinguish the current interaction object.

[0104] The method for displaying virtual content provided in this embodiment includes obtaining a virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects; marking first attribute information of the multiple scene objects on the virtual scene model; obtaining scene object keywords collected by the voice module; obtaining a target scene object from the multiple scene objects based on the scene object keywords; and displaying a virtual scene image of the target scene object and second attribute information in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information. Thus, through the above method, the scene object that the user wants to view can be obtained as the target scene object, and the virtual scene image of the target scene object and the second attribute information are simultaneously displayed in a specified display manner, making the viewing of the target scene object real and vivid and improving the user's viewing experience.

[0105] At the same time, by determining the target scene object with the help of the user's voice content, the determination of the target scene object becomes more user-friendly and intelligent. By introducing the gameplay of the observation glasses to view the virtual image and the real image of the scene object, a more rich interaction method is provided, and thus a more interesting interaction experience is provided.

[0106] Please refer to Figure 13, which shows a flowchart of a method for displaying virtual content provided by another embodiment of the present application. This embodiment provides a method for displaying virtual content, which can be applied to a display device. The method includes:

[0107] Step S410: Obtain a virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects.

[0108] Among them, the specific implementation of step S410 can refer to the relevant description in step S110 in the foregoing embodiment, and will not be elaborated here.

[0109] Step S420: Mark the first attribute information of the multiple scene objects on the virtual scene model.

[0110] Among them, the specific implementation of step S420 can refer to the relevant description in step S120 in the foregoing embodiment, and will not be elaborated here.

[0111] Step S430: Obtain a target scene object from the multiple scene objects.

[0112] Among them, the specific implementation of step S430 can refer to the relevant description in step S130, step S230, step S240, step S330, and step S340 in the foregoing embodiment, and will not be elaborated here.

[0113] Step S440: Obtain the type of the target scene object.

[0114] In this embodiment, in the case of determining the target scene object, in order to provide the effectiveness of virtual content display, the type of the target scene object can be obtained. Here, the type can represent parameters such as the popularity, score, recommendation index, word-of-mouth, and cost performance of the target scene object.

[0115] Step S450: Determine the multimedia introduction information of the target scene object according to the type, and the multimedia introduction information corresponding to different types of target scene objects is different.

[0116] As a way, the multimedia introduction information of the target scene object can be determined according to the type, and the multimedia introduction information corresponding to different types of target scene objects is different, that is, there are significant differences in the display forms corresponding to different types of scene objects.

[0117] Exemplarily, taking the tourism scene as an example, the special effects of popular scenic spots are different from those of unpopular scenic spots (i.e., the display forms), which can help users more effectively understand the information of the scene object, and thus achieve effective virtual content display.

[0118] Optionally, in this embodiment, if the multimedia introduction information (i.e., the second attribute information) displayed for a certain scenic spot is relatively single, and the user happens to be interested in this scenic spot, the user can manually adjust the content range of the second attribute information corresponding to this scenic spot, that is, the user can increase the category of the multimedia introduction information of the current scenic spot. Exemplarily, if the current second attribute information of a certain scenic spot only includes pictures and audio, the user can operate the relevant function buttons to adjust the second attribute information of this scenic spot to pictures + audio + video to meet the personalized viewing needs of the user.

[0119] By allowing different target scene objects (such as buildings) to be displayed in different display forms (such as special effects), the display process of the virtual content can be made more vivid, vivid, and readable, increasing the interest of the virtual content display and enhancing the user viewing experience.

[0120] Step S460: Display the virtual scene image of the target scene object and the second attribute information in a specified display manner, where the second attribute information is different from the first attribute information in terms of type and display form.

[0121] The display device in the embodiment of the present application can also display the ranging information between different scene objects. For example, in a specific application scenario, please refer to Figure 14 , which shows an example diagram of the ranging information between different scene objects provided by the embodiment of the present application. As Figure 14 shown, assuming that the user wants to plan the distance and positional relationship between scenic spot 1 and scenic spot 2, the user can respectively select scenic spot 1 and scenic spot 2 as the target scene objects, and then the ranging information between the two can be displayed on the display screen of the display device to help the user provide an immersive tourism exploration experience.

[0122] The virtual content display method provided in this embodiment obtains a virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects; marks the first attribute information of the multiple scene objects on the virtual scene model; obtains a target scene object from the multiple scene objects; obtains the type of the target scene object; determines the second attribute information of the target scene object according to the type, and the second attribute information corresponding to different types of target scene objects is different; displays the virtual scene image of the target scene object and the second attribute information in a specified display manner, where the second attribute information is different from the first attribute information in terms of type and display form. Thus, through the above method, the scene object that the user wants to view can be obtained as the target scene object, and the virtual scene image of the target scene object and the second attribute information are simultaneously displayed in a specified display manner, making the viewing of the target scene object real and vivid and enhancing the user viewing experience.

[0123] Meanwhile, by allowing different target scene objects (such as buildings) to be displayed in different display forms (such as special effects), the display process of virtual content can be made more vivid, vivid, highly readable, increasing the interest of virtual content display and enhancing the user viewing experience.

[0124] Please refer to Figure 15 , which is a structural block diagram of a virtual content display system provided by an embodiment of the present application. This embodiment provides a virtual content display system 500, and the display system 500 includes a display device 510 and a terminal device 520:

[0125] The terminal device 520 is used to obtain a virtual scene model corresponding to the scene to be viewed, where the scene to be viewed includes multiple scene objects; and mark first attribute information of the multiple scene objects on the virtual scene model.

[0126] The display device 510 is used to obtain a target scene object from the multiple scene objects; and display a virtual scene image and second attribute information of the target scene object in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information.

[0127] In this embodiment, the second attribute information includes the outline of the target scene object, or includes the multimedia introduction information of the target scene object and the outline of the target scene object.

[0128] The display device may include a visual tracking module. As an implementation manner, the display device 510 may specifically be used to obtain pose information collected by the visual tracking module; and obtain a target scene object from the multiple scene objects based on the pose information. Among them, obtaining a target scene object from the multiple scene objects based on the pose information may include: obtaining a corresponding viewing angle range based on the pose information; and taking at least one scene object that falls within the viewing angle range among the multiple scene objects as the target scene object.

[0129] In this embodiment, obtaining a corresponding viewing angle range based on the pose information includes: obtaining a corresponding viewing angle range based on the head movement information of the user; obtaining a corresponding viewing angle range based on the eye movement information of the user; obtaining a corresponding viewing angle range based on the gesture information of the user; and / or obtaining a corresponding viewing angle range based on the touch operation of the user.

[0130] The display device may include a voice module. As another implementation manner, the display device 510 may specifically be used to obtain scene object keywords collected by the voice module; and obtain a target scene object from the multiple scene objects based on the scene object keywords.

[0131] As a way, the display device 510 can also be used to obtain the type of the target scene object before displaying the virtual scene image of the target scene object and the second attribute information in a specified display manner; determine the multimedia introduction information of the target scene object according to the type, and the multimedia introduction information corresponding to different types of target scene objects is different.

[0132] In a specific application scenario, taking a tourism scene as an example, the terminal device can be a device on an observation deck, and this device can be a device with data processing functions such as a PC computer. As an implementation manner, the PC computer can construct a corresponding three-dimensional scene model according to the geographical information data of the scene that the user wants to view (including BIM data, 3D data, map navigation data, etc.), and then mark the name and category of each scenic spot or building (marked in the form of a POI icon) on the three-dimensional scene model, and then obtain the target scene object determined by the user, and then display an image that combines real scene and virtual information of the target scene object with the help of the display device. Compared with the prior art, it can allow the user to participate in the display process of virtual content, enhance the interactive experience during the display process of virtual content, and thus enhance the user experience.

[0133] Optionally, the display device 510 can also be used to display the real scene image of the scene object and display the virtual image of the scene object on the screen of the observation glasses if it is detected that a certain scene object is magnified and viewed through the observation glasses.

[0134] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems and modules can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0135] In several embodiments provided in the present application, the coupling between modules can be electrical, mechanical or other forms of coupling.

[0136] In addition, in each embodiment of the present application, each functional module can be integrated in a processing module, or each module can exist physically alone, or two or more modules can be integrated in one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0137] Please refer to Figure 16, based on the above virtual content display method and system, an embodiment of the present application further provides an electronic device 100 that can execute the virtual content display method described above. The electronic device 100 includes a memory 102 and one or more (only one is shown in the figure) processors 104 that are coupled to each other. The memory 102 and the processor 104 are connected by a communication line. The memory 102 stores a program that can execute the content in the foregoing embodiments, and the processor 104 can execute the program stored in the memory 102.

[0138] Among them, the processor 104 may include one or more processing cores. The processor 104 uses various interfaces and lines to connect various parts within the entire electronic device 100, and by running or executing instructions, programs, code sets, or instruction sets stored in the memory 102, and by calling data stored in the memory 102, it executes various functions of the electronic device 100 and processes data. Optionally, the processor 104 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 104 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 104 and may be implemented separately by a communication chip.

[0139] The memory 102 may include a random access memory (RAM) and may also include a read-only memory. The memory 102 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 102 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the foregoing various embodiments, etc. The data storage area may also store data created during the use of the electronic device 100 (such as phone book, audio and video data, chat record data, etc.).

[0140] Please refer to Figure 17, which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code is stored in the computer-readable medium 600, and the program code can be called by a processor to execute the method described in the above method embodiment.

[0141] The computer-readable storage medium 600 can be an electronic memory such as a flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 600 has a storage space for program code 610 that executes any method step in the above method. These program codes can be read from or written into one or more computer program products. The program code 610 can be compressed in an appropriate form, for example.

[0142] In summary, the embodiments of the present application provide a method, system, electronic device, and storage medium for displaying virtual content. This method is applied to a display device. The method obtains a virtual scene model corresponding to a scene to be viewed, where the scene to be viewed includes multiple scene objects; marks first attribute information of the multiple scene objects on the virtual scene model; obtains a target scene object from the multiple scene objects; and displays a virtual scene image of the target scene object and second attribute information in a specified display manner, where the type and display form of the second attribute information are different from those of the first attribute information. Thus, by the above method, the scene object that the user desires to view can be obtained as the target scene object, and the virtual scene image of the target scene object and the second attribute information can be displayed simultaneously in a specified display manner, making the viewing of the target scene object real and vivid and improving the user's viewing experience.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for displaying virtual content, characterized in that: Applied to a display device, the method comprises: Acquire a virtual scene model corresponding to a scene to be viewed, wherein the scene to be viewed includes a plurality of scene objects; Marking first attribute information of the plurality of scene objects on the virtual scene model; Acquire a target scene object from the multiple scene objects; The virtual scene image and second attribute information of the target scene object are displayed in a specified display manner, wherein the second attribute information is different from the first attribute information in type and display form.

2. The method according to claim 1, characterized in that The second attribute information includes an outline of the target scene object, or includes multimedia introduction information of the target scene object and the outline of the target scene object.

3. The method according to claim 2, characterized in that The display device includes a visual tracking module, which obtains the target scene object from the multiple scene objects, including: Acquire the posture information collected by the visual tracking module; The target scene object is acquired from the multiple scene objects based on the posture information.

4. The method according to claim 3, characterized in that The acquiring the target scene object from the multiple scene objects based on the posture information includes: Acquire a corresponding viewing angle range based on the posture information; At least one scene object among the multiple scene objects that falls within the viewing angle range is taken as a target scene object.

5. The method according to claim 4, characterized in that The obtaining a corresponding viewing angle range based on the posture information includes: Acquire the corresponding viewing angle range based on the user's head movement information; Obtaining the corresponding viewing angle range based on the user's eye movement information; Acquire a corresponding viewing angle range based on the user's gesture information; and / or The corresponding viewing angle range is obtained based on the user's touch operation.

6. The method according to any one of claims 2 to 5, characterized in that: Before displaying the virtual scene image and the second attribute information of the target scene object in a specified display mode, the method further includes: Obtaining the type of the target scene object; The multimedia introduction information of the target scene object is determined according to the type, and different types of target scene objects correspond to different multimedia introduction information.

7. The method according to claim 1, characterized in that The method further comprises: If it is detected that a certain scene object is magnified and viewed through the observation glasses, a virtual image of the scene object is displayed on the screen of the observation glasses at the same time as the real scene image of the scene object is displayed.

8. A virtual content display system, characterized in that: The display system includes a display device and a terminal device, and the display device and the terminal device are communicatively connected: The terminal device is used to obtain a virtual scene model corresponding to a scene to be viewed, wherein the scene to be viewed includes a plurality of scene objects; and Marking first attribute information of the plurality of scene objects on the virtual scene model; The display device is used to obtain a target scene object from the multiple scene objects; as well as The virtual scene image and second attribute information of the target scene object are displayed in a specified display manner, wherein the second attribute information is different from the first attribute information in type and display form.

9. An electronic device, characterized in that: comprising one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images