Virtual reality scene display methods, devices, equipment, media, and program products

By splitting the rotation of the viewpoint direction into single-axis two-dimensional rotation, the problem of slow response speed in virtual reality scenes is solved, and the computational efficiency is improved.

CN120928956BActive Publication Date: 2026-11-17CHINA MOBILE FINANCIAL TECHNOLOGY CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511095259.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-08-06
Publication Date
2026-11-17
Estimated Expiration
2045-08-06

AI Technical Summary

Technical Problem

The current virtual reality scene has a slow response speed, mainly due to the high computational complexity and low efficiency caused by the use of traditional three-dimensional coordinate system transformation mode.

Method used

The rotation in the viewpoint direction is decomposed into multiple two-dimensional rotations on a single principal axis in the target space coordinate system. By decomposing the three-dimensional rotation into a single-axis two-dimensional rotation, the computational complexity is simplified and the computational efficiency is improved.

Benefits of technology

By simplifying computational complexity, the response speed of virtual reality scenes and the efficiency of control information processing are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120928956B_ABST
    Figure CN120928956B_ABST
Patent Text Reader

Abstract

This application provides a method, apparatus, device, medium, and program product for displaying virtual reality scenes, applied in the field of virtual reality technology. The method includes: receiving control information during the display of a current virtual reality (VR) scene; if the control information includes direction information, obtaining the first-view direction corresponding to the current VR scene, the direction information indicating a second-view direction different from the first-view direction; determining a reference coordinate and N positioning coordinates based on the first-view direction and the second-view direction; combining the line connecting the origin and the reference coordinate in the target spatial coordinate system from the reference coordinate to the N rotation trajectories corresponding to the N positioning coordinates to obtain a target rotation trajectory; and controlling the current VR scene to rotate according to the target rotation trajectory. In this method, the computational complexity of the target rotation trajectory is low, and the computational efficiency is high, thereby improving the response speed of the virtual reality scene to control information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and in particular to a method, apparatus, device, medium and program product for displaying virtual reality scenes. Background Technology

[0002] With the development of virtual reality (VR) technology, VR property viewing has become a relatively common application. VR property viewing can be presented on personal computers (PCs) and mobile applications (APPs), providing users with a good experience.

[0003] Currently, when users rotate, zoom in, or switch VR scene images, the VR scene uses a traditional three-dimensional coordinate system transformation mode for display, which results in high computational complexity and low computational efficiency, leading to a slow response speed. Summary of the Invention

[0004] This application provides a method, apparatus, device, medium, and program product for displaying virtual reality scenes, which can solve the problem of slow response speed in virtual reality scenes.

[0005] In a first aspect, embodiments of this application provide a method for displaying a virtual reality scene, the method comprising:

[0006] During the current virtual reality (VR) scene demonstration, control information is received;

[0007] When the control information includes direction information, the first view direction corresponding to the current VR scene is obtained, and the direction information is used to indicate a second view direction that is different from the first view direction;

[0008] Based on the first viewpoint direction and the second viewpoint direction, a reference coordinate and N positioning coordinates are determined. The reference coordinate is the coordinate corresponding to a preset distance in the first viewpoint direction in the target space coordinate system. The target space coordinate system is a space coordinate system established based on the current VR scene. The positioning coordinates are determined based on a single principal axis in the target space coordinate system, the reference coordinate, and the second viewpoint direction. N is a positive integer.

[0009] The line connecting the origin and the reference coordinates in the target spatial coordinate system is rotated from the reference coordinates to the N rotation trajectories corresponding to the N positioning coordinates, and then combined to obtain the target rotation trajectory.

[0010] Control the current VR scene to rotate according to the target rotation trajectory.

[0011] Secondly, embodiments of this application also provide a display device for a virtual reality scene, the device comprising:

[0012] The receiving module is used to receive control information during the current virtual reality (VR) scene display process;

[0013] The first acquisition module is used to acquire the first view direction corresponding to the current VR scene when the control information includes direction information, wherein the direction information is used to indicate a second view direction that is different from the first view direction;

[0014] The first determining module is used to determine a reference coordinate and N positioning coordinates based on the first viewing direction and the second viewing direction. The reference coordinate is the coordinate corresponding to a preset distance in the first viewing direction in the target space coordinate system. The target space coordinate system is a space coordinate system established based on the current VR scene. The positioning coordinates are determined based on a single principal axis in the target space coordinate system, the reference coordinate, and the second viewing direction. N is a positive integer.

[0015] The trajectory compositing module is used to combine the line connecting the origin of the target spatial coordinate system and the reference coordinate from the reference coordinate to the N rotation trajectories corresponding to the N positioning coordinates to obtain the target rotation trajectory;

[0016] The first control module is used to control the current VR scene to rotate according to the target rotation trajectory.

[0017] Thirdly, embodiments of this application also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the virtual reality scene display method as described in the first aspect.

[0018] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the virtual reality scene display method as described in the first aspect.

[0019] Fifthly, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of a method for displaying a virtual reality scene as described in the first aspect.

[0020] In this embodiment, during the display of the current virtual reality (VR) scene, control information is received; if the control information includes direction information, the first viewpoint direction corresponding to the current VR scene is obtained, and the direction information is used to indicate a second viewpoint direction different from the first viewpoint direction; based on the first viewpoint direction and the second viewpoint direction, reference coordinates and N positioning coordinates are determined, where the reference coordinates are the coordinates corresponding to a preset distance in the first viewpoint direction in the target spatial coordinate system, the target spatial coordinate system is a spatial coordinate system established based on the current VR scene, and the positioning coordinates are determined based on a single principal axis in the target spatial coordinate system, the reference coordinates, and the second viewpoint direction, where N is a positive integer; the line connecting the origin of the target spatial coordinate system and the reference coordinates is rotated from the reference coordinates to N rotation trajectories corresponding to the N positioning coordinates and combined to obtain a target rotation trajectory; the current VR scene is controlled to rotate according to the target rotation trajectory. When adjusting the viewpoint direction of the current VR scene, the rotation from the first viewpoint direction to the second viewpoint direction is broken down into multiple rotations on the plane corresponding to a single principal axis in the target space coordinate system. That is, the line connecting the origin and the reference coordinate in the target space coordinate system rotates from the reference coordinate to the N rotation trajectories corresponding to the N positioning coordinates. Therefore, each rotation process of the N rotation trajectories is a two-dimensional rotation, which is easy to understand. Compared with the calculation of three-dimensional coordinate rotation, it simplifies the calculation complexity and improves the calculation efficiency, thereby improving the response speed of the virtual reality scene to control information. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the virtual reality scene display method provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the structure of a system for displaying VR scenes via voice interaction and voice control, as provided in an embodiment of this application.

[0024] Figure 3 This is a schematic diagram of the structure of a virtual reality scene display device provided in an embodiment of this application;

[0025] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] This application provides a method for displaying virtual reality scenes. In the following description, VR scene display is used as an application scenario for VR house viewing to illustrate this application.

[0028] See Figure 1 , Figure 1 This is a flowchart illustrating a method for displaying a virtual reality scene provided in an embodiment of this application. The method includes the following steps:

[0029] Step 101: Receive control information during the current virtual reality (VR) scene display process.

[0030] The current VR scene display could be a panoramic view of a hotel room. Control information is input by the user during the VR room viewing process. User input methods include touching and swiping the device's screen, or speaking to the device to receive voice information.

[0031] Step 102: If the control information includes direction information, obtain the first view direction corresponding to the current VR scene. The direction information is used to indicate a second view direction that is different from the first view direction.

[0032] The first viewpoint direction is the viewpoint direction of the current VR scene. The second viewpoint direction can be a basic direction relative to the first viewpoint direction, or a composite direction relative to the first viewpoint direction. The basic directions include up, down, left, right, front, and back directions relative to the reference position of the viewpoint in the current VR scene. Composite directions are combinations of multiple basic directions, such as left rear, left upper, etc. It can be understood that any second viewpoint direction can be broken down into a combination of basic directions relative to the first viewpoint direction.

[0033] Step 103: Based on the first viewpoint direction and the second viewpoint direction, determine the reference coordinates and N positioning coordinates. The reference coordinates are the coordinates corresponding to a preset distance in the first viewpoint direction in the target space coordinate system. The target space coordinate system is a space coordinate system established based on the current VR scene. The positioning coordinates are determined based on a single principal axis in the target space coordinate system, the reference coordinates, and the second viewpoint direction. N is a positive integer.

[0034] When adjusting the first-view orientation of the current VR scene to the second-view orientation, the second-view orientation can be decomposed into multiple basic orientations relative to the first-view orientation, and rotations can be performed sequentially along these basic orientations. That is, the 3D rotation from the first-view orientation to the second-view orientation is "split" into 2D rotations along each principal axis (X, Y, Z axes), calculated separately, and then combined into a single 3D rotation result. For example, first rotate the object around the X-axis (in the YZ plane) by θ1, then around the Y-axis (in the XZ plane) by θ2, and then around the Z-axis (in the XY plane) by θ3. This way, each operation involves only two coordinates (2D rotations), such as (x, y), (x, z), and (y, z), without needing to process all 3D variables at once.

[0035] The following examples illustrate positioning coordinates:

[0036] Using the reference position of the current VR scene's viewpoint as the origin, establish a target space coordinate system. In the target space coordinate system, the reference coordinates corresponding to the first-view direction A are (a1, b1, c1).

[0037] When the basic direction of direction B in the second perspective is upward compared to direction A, the corresponding positioning coordinates are (a1, -b1, -c1). By pointing the camera in this direction, an upward rotation effect can be achieved.

[0038] When the basic direction of the second perspective direction B is downward compared to the direction A, and the first perspective direction A is downward (that is, the camera is shooting and displaying the panoramic VR image from a downward perspective), the positioning coordinates are (a1, b1, c1). That is, the coordinates remain unchanged, and the downward rotation effect can be achieved by pointing the camera in this direction.

[0039] When the basic direction of direction B in the second perspective is left compared to direction A, the corresponding positioning coordinates are (a1, b1, -c1). At this time, the z-axis position can be moved by σ to convey the movement and rotation effect to the user; that is, the positioning coordinates are (a1, b1, -c1-σ). By pointing the camera in this direction, the leftward rotation effect can be achieved.

[0040] When the basic direction of the second-view direction B is right compared to direction A, and the camera is originally facing right to capture and display the panoramic VR image, the corresponding positioning coordinates are (a1, b1, c1). The coordinates remain unchanged. At this time, the y-axis position is moved by σ to convey the movement and rotation effect to the user; that is, the positioning coordinates are (a1, b1 + σ, c1). Turning the camera in this direction will achieve the effect of rotating to the right.

[0041] When the basic direction of the second-view direction B is forward compared to direction A, and the camera is originally facing forward to shoot and display the panoramic VR image, the corresponding positioning coordinates are (a1, b1, c1). At this time, the x-axis position is moved by σ to convey the movement and rotation effect to the user; that is, the positioning coordinates are (a1+σ, b1, c1). The forward rotation effect can be achieved by pointing the camera in this direction.

[0042] When the basic direction of the second-view direction B is backward compared to the direction A, the corresponding positioning coordinates are (-a1, b1, -c1). At this time, the x-axis position is moved by σ to convey the movement and rotation effect to the user; that is, the positioning coordinates are (-a1-σ, b1, -c1). The camera can be oriented in this direction to achieve the backward rotation effect.

[0043] During the rotation in each of the above directions, you can freely choose whether to perform translation on a single main axis simultaneously to achieve a better performance effect.

[0044] Step 104: Combine the N rotation trajectories of the line connecting the origin and the reference coordinates in the target spatial coordinate system from the reference coordinates to the N positioning coordinates to obtain the target rotation trajectory.

[0045] In this step, N single-axis two-dimensional rotational trajectories are combined into a three-dimensional target rotational trajectory.

[0046] Step 105: Control the current VR scene to rotate according to the target rotation trajectory.

[0047] By rotating the current viewpoint of the VR scene according to the target rotation trajectory, it is possible to change the viewpoint of the current VR scene from the first viewpoint direction to the second rotation direction.

[0048] In this step, when adjusting the viewpoint direction of the current VR scene, the rotation from the first viewpoint direction to the second viewpoint direction is broken down into multiple rotations on the plane corresponding to a single principal axis in the target space coordinate system. That is, the line connecting the origin and the reference coordinate in the target space coordinate system rotates from the reference coordinate to the N rotation trajectories corresponding to the N positioning coordinates. Therefore, each rotation process of the N rotation trajectories is a two-dimensional rotation, which is easy to understand. Compared with the calculation of three-dimensional coordinate rotation, it simplifies the calculation complexity and improves the calculation efficiency, thereby improving the response speed of the virtual reality scene to control information.

[0049] Optionally, the i-th positioning coordinate among the N positioning coordinates is the coordinate corresponding to the preset distance taken in the current view direction corresponding to the current VR scene after the line connecting the reference coordinate and the origin of the target space coordinate system is rotated by the i-th angle in the plane perpendicular to the i-th principal axis of the target space coordinate system, where i is an integer ranging from 1 to N and N is an integer ranging from 1 to 3; the i-th angle and the i-th principal axis are determined based on the first view direction, the second view direction, and the target space coordinate system.

[0050] In this embodiment, the number of positioning coordinates is at most three. Through single-dimensional calculations along the three principal axes in the spatial coordinate system, any direction in the spatial coordinate system can be obtained. When N is greater than 1, the i-th principal axis may include the 1st principal axis and the 2nd principal axis, or it may also include the 3rd principal axis. The 1st, 2nd, and 3rd principal axes are all different. For example, the first viewpoint direction is the x-axis direction of the target spatial coordinate system, with reference coordinates of (1, 0, 0). The coordinates corresponding to the second viewpoint direction are (1, 1, 1). The second viewpoint direction relative to the first viewpoint direction can then be decomposed into rotation in the xy-plane and rotation in the xz-plane. This yields two positioning coordinates and their corresponding rotational trajectories. Combining these two coordinates yields the target rotational trajectory.

[0051] In this embodiment, by decomposing the three-dimensional rotation into a two-dimensional composite rotation along a single axis, the complex three-dimensional operation is simplified into a simple three-dimensional operation of order three. This reduces the overall computational complexity, improves computational efficiency, and thus improves the response speed of the VR scene.

[0052] Optionally, before controlling the current VR scene to rotate according to the target rotation trajectory, the method further includes:

[0053] If the control information also includes scene information, and the target VR scene indicated by the scene information is different from the current VR scene, the material of the target VR scene is obtained from the preset material library, and the first position coordinates and third view direction of the target VR scene are obtained in advance.

[0054] The materials of the target VR scene are displayed, and the reference position of the viewpoint of the displayed target VR scene is the position indicated by the first position coordinate in the target VR scene, and the viewpoint direction of the displayed target VR scene is the third viewpoint direction.

[0055] In this embodiment, the control information also includes scene information. In a VR house viewing scenario, different scene information can be understood as different house types. It should be noted that when the control information includes both scene information and direction information, the scene switching corresponding to the scene information is executed first, and then the viewpoint direction is rotated in the switched scene.

[0056] For example, obtain the labeled coordinates (x, y, z) (first position coordinates) of room X (target VR scene). Whether it is global coordinates or spherical coordinates, it is a three-dimensional coordinate system. Set the orientation (view direction) of the panoramic camera corresponding to room X to (x, 0, 0), with the x-axis as the main orientation. Subsequent position, direction, and speed switching are all based on this direction.

[0057] In this step, VR scenes can be switched using scene information, making the control methods of VR scenes more diverse and improving the user experience.

[0058] Optionally, after displaying the materials of the target VR scene, the method further includes:

[0059] If the control information also includes position information, and the second position coordinates indicated by the position information are inconsistent with the first position coordinates, and the position indicated by the second position coordinates exists in the target VR scene, the reference position of the viewpoint of the target VR scene is adjusted based on the second position coordinates. The adjusted reference position of the viewpoint of the target VR scene is the position indicated by the second position coordinates in the target VR scene.

[0060] In this embodiment, location information in a VR house viewing scenario can be understood as a specific location within a room. When the control information simultaneously includes scene information, location information, and direction information, the scene switching corresponding to the scene information is executed first, followed by the switching of the reference position of the VR scene's viewpoint corresponding to the location information, and finally, the rotation of the viewpoint direction is performed. This embodiment's method further enhances the diversity of VR scene control methods, improving the user experience.

[0061] Optionally, during the current virtual reality (VR) scene display, after receiving control information, the method further includes:

[0062] If the control information also includes speed information, the current playback speed during the movement of the screen in the current VR scene is obtained, and the screen movement includes translation or rotation;

[0063] The current playback speed is adjusted based on the speed information to obtain the adjusted playback speed, wherein the speed information is either speeding up or slowing down.

[0064] Control the current VR scene to move the screen according to the adjusted playback speed.

[0065] In this embodiment, the speed information can be a string type, a number type, or other types. The difference between string and number types is that the speed information is converted from Chinese characters into corresponding numerical codes starting from 1, such as 1 representing low speed, 2 representing low-medium speed, 3 representing medium speed, 4 representing medium-high speed, and 5 representing high speed, etc.

[0066] When switching speeds, 1 represents low speed, indicated by 1 displacement; 2 represents medium-low speed, indicated by 2 displacements; 3 represents medium speed, indicated by 3 displacements; 4 represents medium-high speed, indicated by 4 displacements; and 5 represents high speed, indicated by 5 displacements.

[0067] Upon receiving speed information, the following calculations are performed:

[0068] <1> Get the current playback speed v1;

[0069] <2> Speed ​​information is obtained and used to indicate the user's speed intention—whether to increase or decrease the speed. If it is to increase, then... <3> If you reduce speed, turn <4> ;

[0070] <3> At this point, the user needs to increase the speed. Based on the five speed levels corresponding to speed v1, the user can increase the speed by one level. For example, if the speed was originally 2 (low to medium speed), the user can increase the speed to 3 (medium speed).

[0071] change <5> ;

[0072] <4> At this point, the user needs to slow down. Based on one of the five speed levels corresponding to speed v1, the speed is reduced by one level. That is, if the original speed was 2 (medium-low speed), the speed is now increased to 1 (low speed).

[0073] change <5> ;

[0074] <5> End calculation.

[0075] The adjusted playback speed will be reflected in the forward or backward speed of a certain axis of the three-dimensional coordinate system corresponding to the user's current viewing direction, as well as the speed of the VR scene's rotation.

[0076] The method in this embodiment further enhances the diversity of control methods for VR scenes, thereby improving the user experience.

[0077] Optionally, during the current virtual reality (VR) scene display process, receiving control information includes:

[0078] During the current virtual reality (VR) scene demonstration, control voice commands are received;

[0079] The control voice is subjected to intent recognition to determine the control information.

[0080] In this embodiment, control information can be obtained through the user's voice. After receiving the user's voice, the VR scene can be controlled, that is, the VR scene can be displayed through voice interaction. In this embodiment, compared with the method of controlling the VR scene by manual operation, voice control is more convenient and faster, and can improve the user experience.

[0081] like Figure 2 As shown, the system for displaying VR scenes through voice interaction and voice control includes four modules: a front-end digital human voice interaction module, a VR display module, and a back-end AI module and business module.

[0082] The digital human voice interaction module is responsible for interacting with the user. Users can press and hold the voice input box to speak and input commands to control VR rotation. The commands are then passed to the backend AI module, which performs speech recognition and intent recognition to understand and recognize the VR controls. It also identifies the voice (male or female) for input to the business module. The business module includes the storage of VR materials (including VR materials and corresponding annotation data), storage of user viewing styles, storage of user characteristics (including but not limited to voice), configuration of new user cold start strategies, statistics of viewers, and rotation control. The VR material annotations include at least three categories: room type, direction, and location. Room type includes options like "double room" or "twin room"; direction includes forward, backward, left, right, down, and up; and location includes different functional areas within the room, such as the living room, bathroom, bed, and windows. In addition to the functional areas, panoramic coordinates of each functional area are also annotated.

[0083] After the AI ​​module transmits the user's intent to control VR to the business module, the business module controls the VR material through the rotation control program to complete the VR rotation corresponding to the user's intent, so that the user can conveniently and intelligently view the VR material in the VR display module on the front end.

[0084] The following example illustrates how to display a VR scene using voice interaction and voice control:

[0085] Step 1: Obtain VR materials and statistics; When a user logs into the digital human page and views VR materials for a specific hotel during the hotel booking process, the business module returns the required VR materials to the digital human page and increments the number of viewers and the number of viewers currently viewing by 1, so that the two fields of "viewed" and "currently viewing" can be synchronously returned to the digital human page for display.

[0086] Step 2: VR control voice command input; The user presses and holds the voice input button on the digital human page and speaks the control command for the hotel room VR. The digital human page records the voice and forms an audio file, which is then submitted to the server-side AI module through the interface.

[0087] Step 3: Speech Recognition; The AI ​​module receives the audio file from Step 2, performs speech recognition to identify the text corresponding to the audio file, and obtains the text string; then it performs voice recognition to determine whether it is a male or female voice, and obtains the voice result.

[0088] The text string (text) and the voice result (voice) are sent to the AI ​​intent recognition module.

[0089] Step 4: Intent Recognition; The AI ​​module receives the text string (text) and the voice result (voice). It performs intent recognition based on the text string, primarily extracting words related to room type, direction, location, and speed to identify the user's intent. The room type, direction, and location information are identical to the VR annotations for hotel rooms, and speed refers to the automatic playback speed of the VR content.

[0090] If the text string contains words related to room type, it means that the user intends to go to that room type to view the corresponding VR content.

[0091] If the text string contains words related to direction, it indicates that the user intends to view VR content in the direction corresponding to the current room.

[0092] If the text string contains location-related words, it indicates that the user intends to view VR content for the corresponding location in the current room.

[0093] If the text string contains words related to speed, it indicates that the user intends to speed up or slow down the viewing speed of the VR content in the current room.

[0094] The words, intention, and voice result from step three are sent to the business module.

[0095] Step 5: Store user viewing styles and characteristics, which will be described in detail in subsequent embodiments.

[0096] Step Six: VR Material Rotation Control, which is consistent with the above embodiment of controlling the display of VR scenes based on control information (scene information, position information, direction information, speed information).

[0097] Step 7: VR demonstration.

[0098] The VR display module located at the front desk receives the VR rotation control parameters (including the target rotation trajectory) from step six, performs the corresponding parameter conversion (direction, position, room type), and presents it to the user's mobile phone.

[0099] Optionally, before receiving control voice, the method further includes:

[0100] When the current VR scene is displayed as the VR scene that the target user is viewing for the first time, the voice information of the target user is collected, and the target user is the user who makes the control voice.

[0101] Obtain a list of preset playback speeds corresponding to the timbre information;

[0102] The average speed obtained by rounding up the speeds in the preset playback speed list is taken as the current playback speed.

[0103] In this embodiment, when a user first enters the first VR content, the initial VR viewing speed is controlled by a new user cold start strategy. This strategy is calculated based on user characteristic data, and the calculation logic is as follows:

[0104] <1> Obtain the voice timbre during the user's speech recognition process, i.e., whether it is a male or female voice; voice=1 indicates a male voice, and voice=0 indicates a female voice.

[0105] <2> The system retrieves voice value data from user characteristics and viewing styles from the business module, and obtains a corresponding playback speed list based on the voice value data. For example, when voice = 1, it retrieves a playback speed data list List for male voices, where the playback speed list is a historical playback speed list.

[0106] <3> Based on the data in the playback speed data list, the average value is obtained by rounding up. This value corresponds to one of the five playback speeds, which is the default playback speed for new users, i.e., the current playback speed when the target user views the VR scene for the first time.

[0107] In this embodiment, when a target user is viewing a VR scene for the first time, an initial default playback speed is set for the user based on the user's voice information. This makes the default playback speed more accurate (more in line with the target user's preferences) and improves the user experience.

[0108] Optionally, after collecting the timbre information of the target user, the method further includes:

[0109] The timbre information is bound to the control information to generate the viewing style information of the target user;

[0110] If the target user views the VR scene again and no new control voice is received from the target user, the VR scene will be displayed based on the control information in the viewing style information.

[0111] The steps in this embodiment can be regarded as step five of the above-mentioned VR scene display through voice interaction and voice control, storing user viewing style and characteristics: After the business module receives words, intention and voice result, it mainly performs user viewing style and characteristics storage and VR material rotation control, wherein this step performs user viewing style and characteristics storage.

[0112] User characteristic storage primarily stores, but is not limited to, the correspondence between a user's system ID and voice timbre. For example, ID = 10001, voice timbre = 1, where 1 represents a male voice and 0 represents a female voice. User viewing style primarily stores the user's habitual VR content viewing routes and speeds. Viewing routes mainly refer to the user's positional changes after entering a certain type of room (e.g., from the living room to the window to the bathroom), and the viewing speed and direction at different positions. This part of the viewing position and corresponding viewing direction and speed data is recorded from the moment the user enters the VR content until the user finishes watching the VR content. The recording method for the main fields is as follows:

[0113] User ID, ID = 10001;

[0114] Room type, type=1 (different codes represent different room types, such as 1 for a king room, 2 for a twin room, 3 for a deluxe king room, 4 for a deluxe twin room, etc.);

[0115] Room position, orientation, and velocity are organized in the form of triples, that is, (room position, orientation, and velocity) represent the entire process of a user watching VR in multiple triples.

[0116] The room location field can be a string, or it can be a number or other types. The difference between string and number types is that the location information is converted from Chinese characters into corresponding numerical codes starting from 1, such as 1 for hall, 2 for window, and 3 for bathroom.

[0117] The direction field can be a string, or it can be a number or other types. The difference between string and number types is that the direction information is converted from Chinese characters to corresponding numerical codes starting from 1, such as 1 for east, 2 for south, 3 for west, and 4 for north.

[0118] The speed field can be a string, or a number or other types. The difference between string and number types is that the speed information is converted from Chinese characters into corresponding numerical codes starting from 1, such as 1 representing low speed, 2 representing low-medium speed, 3 representing medium speed, 4 representing medium-high speed, and 5 representing high speed, etc.

[0119] Through the above steps, the viewing style and characteristics of the target user can be stored. When the target user watches the VR scene display again, the VR scene can be displayed based on the stored viewing style information. The viewing style can conform to the user's habits, thereby improving the user experience.

[0120] In an optional implementation, in a scenario where VR scenes are displayed via voice interaction and voice control, the following steps can be performed upon receiving control information:

[0121] When the AI ​​module's speech recognition and intent recognition determine that the user wants to switch to a specific room X, the following steps are performed:

[0122] Check if room X exists in the hotel's VR material library. If it does not exist, notify the user that the room is not available. The interaction method is not limited to pop-up windows, text prompts, digital human voice broadcasts, etc. If room X exists, then perform the process of controlling the VR scene based on scene information as described in the above embodiment.

[0123] When the AI ​​module's speech recognition and intent recognition determine that the user wants to switch to the Y position of a certain room X, the following steps are performed:

[0124] Check if the Y position (e.g., window) of room X exists in the hotel's VR material library. If it does not exist, inform the user that there is no corresponding position for the room. The interaction method is not limited to pop-up, text prompt, digital human voice broadcast, etc. If the Y position of room X exists, then perform the process of controlling the VR scene based on the position information in the above embodiment.

[0125] When the AI ​​module's speech recognition and intent recognition obtain the directional information that the user wants to switch the viewing direction, the following steps are performed:

[0126] The basic validity of the directional information is determined by whether it falls within the basic directions of up, down, left, back, front, and back. If not, the directional description is incorrect, and the user is not informed that there is no corresponding direction for the room. Interaction methods are not limited to pop-ups, text prompts, or digital human voice announcements. The core validity of the directional information is further verified after confirming that the directional information includes the basic directions of up, down, left, back, front, and back. It then checks whether it includes composite directional descriptions, such as left and back. If there are opposite descriptions (such as up and down, left and right, front and back) or three directions, the directional description is incorrect, and the user is informed that there is no corresponding direction for the room. Interaction methods are not limited to pop-ups, text prompts, or digital human voice announcements. If the above validity checks are passed, the process of controlling the VR scene based on the directional information, as shown in the above embodiment, will be executed.

[0127] See Figure 3 , Figure 3 This is a schematic diagram of the structure of a virtual reality scene display device provided in an embodiment of this application. For example... Figure 3 As shown, the device 300 includes:

[0128] The receiving module 301 is used to receive control information during the current virtual reality (VR) scene display process;

[0129] The first acquisition module 302 is used to acquire the first view direction corresponding to the current VR scene when the control information includes direction information, wherein the direction information is used to indicate a second view direction that is different from the first view direction;

[0130] The first determining module 303 is used to determine a reference coordinate and N positioning coordinates based on the first viewing direction and the second viewing direction. The reference coordinate is the coordinate corresponding to a preset distance in the first viewing direction in the target space coordinate system. The target space coordinate system is a space coordinate system established based on the current VR scene. The positioning coordinates are determined based on a single principal axis in the target space coordinate system, the reference coordinate, and the second viewing direction. N is a positive integer.

[0131] The trajectory composite module 304 is used to combine the line connecting the origin of the target spatial coordinate system and the reference coordinate from the reference coordinate to the N rotation trajectories corresponding to the N positioning coordinates to obtain the target rotation trajectory;

[0132] The first control module 305 is used to control the current VR scene to rotate according to the target rotation trajectory.

[0133] Optionally, the i-th positioning coordinate among the N positioning coordinates is the coordinate corresponding to the preset distance taken in the current view direction corresponding to the current VR scene after the line connecting the reference coordinate and the origin of the target space coordinate system is rotated by the i-th angle in the plane perpendicular to the i-th principal axis of the target space coordinate system, where i is an integer ranging from 1 to N and N is an integer ranging from 1 to 3; the i-th angle and the i-th principal axis are determined based on the first view direction, the second view direction, and the target space coordinate system.

[0134] Optionally, the device 300 further includes:

[0135] The second acquisition module is used to acquire materials of the target VR scene from a preset material library when the control information further includes scene information and the target VR scene indicated by the scene information is different from the current VR scene, and to acquire the first position coordinates and third view direction of the target VR scene that are pre-marked in the target VR scene.

[0136] The second control module is used to display the materials of the target VR scene. The reference position of the viewpoint of the displayed target VR scene is the position indicated by the first position coordinate in the target VR scene, and the viewpoint direction of the displayed target VR scene is the third viewpoint direction.

[0137] Optionally, the device 300 further includes:

[0138] The first adjustment module is configured to adjust the reference position of the viewpoint of the target VR scene based on the second position coordinate when the control information further includes position information, the second position coordinate indicated by the position information is inconsistent with the first position coordinate, and the position indicated by the second position coordinate exists in the target VR scene. The adjusted reference position of the viewpoint of the target VR scene is the position indicated by the second position coordinate in the target VR scene.

[0139] Optionally, the device 300 further includes:

[0140] The third acquisition module is used to acquire the current playback speed during the screen movement process of the current VR scene when the control information also includes speed information, wherein the screen movement includes translation or rotation.

[0141] The second adjustment module is used to adjust the current playback speed based on the speed information to obtain the adjusted playback speed, wherein the speed information is either speeding up or speeding up.

[0142] The third control module is used to control the current VR scene to move the screen according to the adjusted playback speed.

[0143] Optionally, the receiving module 301 is also used for:

[0144] During the current virtual reality (VR) scene demonstration, control voice commands are received;

[0145] The control voice is subjected to intent recognition to determine the control information.

[0146] Optionally, the device 300 further includes:

[0147] The acquisition module is used to acquire the voice information of the target user when the current VR scene is displayed as the first VR scene view of the target user. The target user is the user who issued the control voice.

[0148] The fourth acquisition module is used to acquire a list of preset playback speeds corresponding to the timbre information;

[0149] The speed determination module is used to take the average speed obtained by rounding up the speeds in the preset playback speed list as the current playback speed.

[0150] Optionally, the device 300 further includes:

[0151] A binding module is used to bind the timbre information with the control information to generate the viewing style information of the target user;

[0152] The fourth control module is used to display the VR scene based on the control information in the viewing style information when the target user watches the VR scene display again and no new control voice is received from the target user.

[0153] The virtual reality scene display device 300 provided in this application embodiment is capable of implementing... Figure 1 The entire technical process of the virtual reality scene display method shown is used to achieve the same technical effect, and will not be repeated here to avoid repetition.

[0154] The virtual reality scene display device 300 in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. Non-mobile electronic devices can also be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not specifically limit the specific devices.

[0155] Optionally, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401 and a memory 402. The memory 402 stores a program or instructions that can run on the processor 401. When the program or instructions are executed by the processor 401, they implement the various steps of the above-described virtual reality scene display method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0156] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0157] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described virtual reality scene display method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0158] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1The various processes of the virtual reality scene display method embodiment shown are all able to achieve the same technical effect, and will not be described again here to avoid repetition.

[0159] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0161] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for displaying a virtual reality scene, characterized in that, The method includes: During the current virtual reality (VR) scene demonstration, control information is received; When the control information includes direction information, the first view direction corresponding to the current VR scene is obtained, and the direction information is used to indicate a second view direction that is different from the first view direction; Based on the first viewpoint direction and the second viewpoint direction, a reference coordinate and N positioning coordinates are determined. The reference coordinate is the coordinate corresponding to a preset distance in the first viewpoint direction in the target space coordinate system. The target space coordinate system is a space coordinate system established based on the current VR scene. The positioning coordinates are determined based on a single principal axis in the target space coordinate system, the reference coordinate, and the second viewpoint direction. N is a positive integer. The line connecting the origin and the reference coordinates in the target spatial coordinate system is rotated from the reference coordinates to the N rotation trajectories corresponding to the N positioning coordinates, and then combined to obtain the target rotation trajectory. Control the current VR scene to rotate according to the target rotation trajectory; The i-th positioning coordinate among the N positioning coordinates is the coordinate corresponding to the preset distance taken in the current view direction corresponding to the current VR scene after the line connecting the reference coordinate and the origin of the target space coordinate system is rotated by the i-th angle in the plane perpendicular to the i-th principal axis of the target space coordinate system. i is an integer ranging from 1 to N, and N is an integer ranging from 1 to 3. The i-th angle and the i-th principal axis are determined based on the first view direction, the second view direction, and the target space coordinate system.

2. The method according to claim 1, characterized in that, Before controlling the current VR scene to rotate according to the target rotation trajectory, the method further includes: If the control information also includes scene information, and the target VR scene indicated by the scene information is different from the current VR scene, the material of the target VR scene is obtained from the preset material library, and the first position coordinates and third view direction of the target VR scene are obtained in advance. The materials of the target VR scene are displayed, and the reference position of the viewpoint of the displayed target VR scene is the position indicated by the first position coordinate in the target VR scene, and the viewpoint direction of the displayed target VR scene is the third viewpoint direction.

3. The method according to claim 2, characterized in that, After displaying the materials of the target VR scene, the method further includes: If the control information also includes position information, and the second position coordinates indicated by the position information are inconsistent with the first position coordinates, and the position indicated by the second position coordinates exists in the target VR scene, the reference position of the viewpoint of the target VR scene is adjusted based on the second position coordinates. The adjusted reference position of the viewpoint of the target VR scene is the position indicated by the second position coordinates in the target VR scene.

4. The method according to any one of claims 1 to 3, characterized in that, During the current virtual reality (VR) scene demonstration, after receiving control information, the method further includes: If the control information also includes speed information, the current playback speed during the movement of the screen in the current VR scene is obtained, and the screen movement includes translation or rotation; The current playback speed is adjusted based on the speed information to obtain the adjusted playback speed, wherein the speed information is either speeding up or slowing down. Control the current VR scene to move the screen according to the adjusted playback speed.

5. The method according to claim 4, characterized in that, During the current virtual reality (VR) scene display process, receiving control information includes: During the current virtual reality (VR) scene demonstration, control voice commands are received; The control voice is subjected to intent recognition to determine the control information.

6. The method according to claim 5, characterized in that, Before receiving control voice commands, the method further includes: When the current VR scene is displayed as the VR scene that the target user is viewing for the first time, the voice information of the target user is collected, and the target user is the user who makes the control voice. Obtain a list of preset playback speeds corresponding to the timbre information; The average speed obtained by rounding up the speeds in the preset playback speed list is taken as the current playback speed.

7. The method according to claim 6, characterized in that, After collecting the timbre information of the target user, the method further includes: The timbre information is bound to the control information to generate the viewing style information of the target user; If the target user views the VR scene again and no new control voice is received from the target user, the VR scene will be displayed based on the control information in the viewing style information.

8. A display device for virtual reality scenes, characterized in that, The device includes: The receiving module is used to receive control information during the current virtual reality (VR) scene display process; The first acquisition module is used to acquire the first view direction corresponding to the current VR scene when the control information includes direction information, wherein the direction information is used to indicate a second view direction that is different from the first view direction; The first determining module is used to determine a reference coordinate and N positioning coordinates based on the first viewing direction and the second viewing direction. The reference coordinate is the coordinate corresponding to a preset distance in the first viewing direction in the target space coordinate system. The target space coordinate system is a space coordinate system established based on the current VR scene. The positioning coordinates are determined based on a single principal axis in the target space coordinate system, the reference coordinate, and the second viewing direction. N is a positive integer. The trajectory compositing module is used to combine the line connecting the origin of the target spatial coordinate system and the reference coordinate from the reference coordinate to the N rotation trajectories corresponding to the N positioning coordinates to obtain the target rotation trajectory; The first control module is used to control the current VR scene to rotate according to the target rotation trajectory; The i-th positioning coordinate among the N positioning coordinates is the coordinate corresponding to the preset distance taken in the current view direction corresponding to the current VR scene after the line connecting the reference coordinate and the origin of the target space coordinate system is rotated by the i-th angle in the plane perpendicular to the i-th principal axis of the target space coordinate system. i is an integer ranging from 1 to N, and N is an integer ranging from 1 to 3. The i-th angle and the i-th principal axis are determined based on the first view direction, the second view direction, and the target space coordinate system.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for displaying a virtual reality scene as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for displaying a virtual reality scene as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the method for displaying a virtual reality scene as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method, device and equipment for realizing information processing based on virtual scene and storage medium

    CN108762492A

  • Virtual reality equipment and sight line following method for object in virtual reality scene

    CN112817453A