Interaction method and device, electronic equipment and storage medium

Abstract

The invention relates to an interaction method and device, electronic equipment and a storage medium. The method comprises the following steps: displaying a video playing window in an augmented reality space; the video playing window is used for displaying image content of a local area in the VR video image; and in response to an adjustment operation on the VR video picture, keeping the position of the video playing window fixed in the augmented reality space, adjusting a local area of the VR video image displayed by the video playing window, the picture displayed by the adjusted video playing window being not completely the same as the picture displayed by the video playing window before the adjustment in content, and if the picture is not completely the same as the picture displayed by the video playing window before the adjustment in content, displaying the VR video image in the augmented reality space. The essence of the invention is to provide a method for allowing a user to adjust the viewing angle of the VR video according to own needs. The method can meet the personalized watching requirements of the user.

Description

Technical Field

[0001] The present disclosure relates to the technical field of extended reality, and in particular to an interaction method, device, electronic device and storage medium. Background Art

[0002] VR video is a type of video shot with a dedicated 360° or 180° VR camera based on extended reality technology. Its core feature is the ability to capture all-round image information and provide viewers with a field of view without blind spots. Shared Space is an experience mode provided by extended reality devices. In this experience mode, users can experience the fusion of the physical world and virtual content at the same time. This experience mode allows users to interact with the surrounding environment and other people without being completely isolated in a virtual environment. Shared Space is very suitable for situations that require attention to both the real world and digital content, such as collaborative work, social interaction, and enhanced real-time applications.

[0003] In the shared space mode, when a user needs to watch a VR video, the system will use the video playback window to present the VR video. However, due to the size and display characteristics of the video playback window, the video playback window can only display a local area image of the VR video, and cannot present the entire VR screen at once. This results in the viewing angle currently presented by the system may not meet the user's personalized needs. Therefore, how to help users find a viewing angle that meets their needs is an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an interaction method, an apparatus, an electronic device and a storage medium.

[0005] In a first aspect, the present disclosure provides an interaction method, comprising:

[0006] Displaying a video playback window in the extended reality space; the video playback window is used to display the image content of a local area in the VR video image;

[0007] In response to the adjustment operation of the VR video picture, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is adjusted. The picture displayed in the video playback window after the adjustment is not completely the same as the picture content displayed in the video playback window before the adjustment.

[0008] In a second aspect, the present disclosure further provides an interactive device, including:

[0009] A display module, used to display a video playback window in an extended reality space; the video playback window is used to display image content of a local area in a VR video image;

[0010] The adjustment module is used to respond to the adjustment operation of the VR video picture, keep the position of the video playback window fixed in the extended reality space, and adjust the local area of ​​the VR video image displayed in the video playback window. The picture displayed in the video playback window after the adjustment is not completely the same as the picture content displayed in the video playback window before the adjustment.

[0011] In a third aspect, the present disclosure further provides an electronic device, the electronic device comprising:

[0012] one or more processors;

[0013] A storage device for storing one or more programs;

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method as described above.

[0015] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned interaction method when executed by a processor.

[0016] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0017] The technical solution provided by the embodiment of the present disclosure is to display a video playback window in an extended reality space; the video playback window is used to display the image content of a local area in a VR video image; in response to the adjustment operation of the VR video image, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is adjusted. The image displayed in the video playback window after the adjustment is not completely the same as the image content displayed in the video playback window before the adjustment. Its essence is to provide a method that allows users to adjust the viewing angle of VR videos according to their own needs. It can meet the personalized viewing needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 A flowchart of an interactive method provided by an embodiment of the present disclosure;

[0021] Figure 2-Figure 5 Several examples of interaction principles provided for embodiments of the present disclosure;

[0022] Figure 6 is a schematic diagram of the structure of an interactive device in an embodiment of the present disclosure;

[0023] Figure 7 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0026] Figure 1 This is a flowchart of an interaction method provided in an embodiment of the present disclosure. This embodiment is applicable to situations where interaction is performed in a client. The method can be executed by an interaction device, which can be implemented in software and / or hardware, and can be configured in an extended reality device.

[0027] The extended reality devices described in this application may include but are not limited to the following types:

[0028] The computer-side extended reality device uses the PC to perform related calculations and data output of the extended reality function, and the external computer-side extended reality device uses the data output by the PC to achieve the extended reality effect.

[0029] Mobile extended reality devices support the setting of mobile terminals (such as smart phones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations of the extended reality function and outputs data to the mobile extended reality device, such as watching extended reality videos through the mobile terminal's APP.

[0030] The all-in-one extended reality device has a processor for performing relevant calculations of virtual functions, and thus has independent extended reality input and output functions. It does not need to be connected to a PC or mobile terminal and has a high degree of freedom in use.

[0031] like Figure 1 As shown, the method may specifically include:

[0032] S110. Display a video playback window in the extended reality space; the video playback window is used to display image content of a local area in the VR video image.

[0033] The extended reality space can be, for example, a virtual scene created with the help of an environmental image when the extended reality device is running. The virtual scene can be a simulated environment of the real world, a semi-simulated and semi-fictional virtual scene, or a purely fictional virtual scene. The extended reality space can include the sky, land, ocean, etc. The land can include environmental elements such as deserts and cities, and users can move in the extended reality space. The environmental image is equivalent to the desktop wallpaper displayed after the computer is turned on.

[0034] The video playback window may be, for example, an area in the extended reality space for playing VR videos at 360°, 180°, or other angles. In practice, the video playback window may be, for example, an area in a control panel for playing VR videos.

[0035] VR video corresponds to the ball model. The ball model can be a three-dimensional geometric model used to carry VR video images. It is an idealized sphere, and its surface is used as a mapping carrier for VR video image content. There is a correspondence between each point on the surface of the ball model and the pixel points in the VR video image. This correspondence can be determined by spherical projection methods such as equirectangular projection or azimuthal equal-area projection AEP (Adjusted equal-area projection). With the correspondence between the points on the surface of the ball model and the pixel points in the VR video image, the VR image can be accurately "attached" to the surface of the ball model.

[0036] Use the video playback window to play the VR video. The video playback window is similar to a window. Users watch the VR video through the video playback window, which is similar to the user "looking through" the window to observe the scenery outside. For details, see Figure 2 , the extended reality space includes the user ( Figure 2 D1 represents the user's position in the extended reality space), video playback windows A1A2A3A4 and the ball model corresponding to the VR video. Figure 2 In the figure, rings L1 and L2 are two rings on the ball model corresponding to the VR video. The user at position D1 can watch the VR video image "attached" to the ball model through the video playback window A1A2A3A4. And, due to the limitation of the video playback window A1A2A3A4, the user can only see the image of the local area "attached" to the surface of the ball model. For example, when the user is in Figure 2 When the position D1 is reached, only the image "attached" to the surface of the ball model in the area B1B2B3B4 can be seen. In other words, when the VR video is played in the video playback window, the video playback window can only display the image of a specific area of ​​the VR video image.

[0037] It should be emphasized that in the extended reality space, the ball model is invisible to the user.

[0038] S120. In response to the adjustment operation on the VR video picture, the position of the video playback window in the extended reality space is kept fixed, and a local area of ​​the VR video image displayed in the video playback window is adjusted. The picture displayed in the video playback window after the adjustment is not completely the same as the picture displayed in the video playback window before the adjustment.

[0039] The adjustment operation of the VR video screen may be, for example, an operation indicating that the user wishes to change the local area of ​​the VR video image displayed in the video playback window. Changing the local area may, for example, mean that the original video playback window displays the screen content of the local area 1 of the VR video image, and after adjustment, the video playback window displays the screen content of the local area 2 of the VR video image. Local area 1 and local area 2 may be completely different, that is, local area 1 and local area 2 do not overlap with each other; or local area 1 and local area 2 may partially overlap. For example, local area 1 includes part or all of local area 2, as well as some other areas; it is also possible that local area 2 includes part or all of local area 1, as well as some other areas.

[0040] There are many methods for implementing this step, and this application does not limit this. Exemplarily, the implementation method of this step may include: in response to the adjustment operation of the VR video screen, determining the target posture information of the ball model; the target posture information of the ball model includes the target position information and / or target posture information of the ball model in the extended reality space; based on the target posture information of the ball model, adjusting the position and / or posture of the ball model in the extended reality space.

[0041] The pose information of the ball model may be, for example, the position and / or posture of the ball model in the extended reality space. The target pose information of the ball model may be, for example, the position and / or posture to which the ball model needs to be adjusted in order to achieve the purpose of "the picture displayed in the video playback window after adjustment is not completely the same as the picture displayed in the video playback window before adjustment".

[0042] In the technical solution of the present application, the correspondence between the points on the surface of the ball model and the pixels in the VR video image remains unchanged. This means that when the posture of the ball model changes, the local area of ​​the VR video image displayed in the video playback window will change. For example, see Figure 3 , at a certain moment, if the user is at position D1, the image "attached" to the surface of the ball model in area B1B2B3B4 can be seen through the video playback window. It can be considered that the reason why the user can see the image "attached" to the surface of the ball model in area B1B2B3B4 is because the image of the ball model in area B1B2B3B4 is facing the video playback window. Figure 2 The ball model rotates in the direction of F. At another moment, the image "attached" to the C1C2C3C4 area of ​​the ball model will be "rotated" to face the video playback window and be seen by the user.

[0043] By setting a response to the adjustment operation of the VR video screen, the target position information of the ball model is determined. Its essence is to adjust the local area of ​​the VR video image displayed in the video playback window by adjusting the position and / or posture of the ball model.

[0044] Further, in some embodiments, the adjustment operation on the VR video screen includes a movement operation on the VR video screen, and "determining the target posture information of the ball model in response to the adjustment operation on the VR video screen" may include: based on the movement trajectory of the VR video screen, determining the moving distance of the moving trajectory in the horizontal direction and the moving distance in the vertical direction; based on the moving distance of the moving trajectory in the horizontal direction, determining the target yaw angle of the ball model; based on the moving distance of the moving trajectory in the vertical direction, determining the target pitch angle of the ball model; determining that the target posture information of the ball model includes the target yaw angle and target pitch angle of the ball model.

[0045] The movement operation of the VR video screen may be, for example, an operation indicating that the user wishes to move the VR video screen upward, downward, left, or right. Specifically, the user may slide or drag the video playback window on the plane or curved surface using a handle or gesture.

[0046] The movement trajectory of the VR video screen may be, for example, the movement trajectory recorded during the process of moving the VR video screen.

[0047] Determining the target yaw angle of the ball model based on the moving distance of the moving trajectory in the horizontal direction, for example, may include: pre-constructing a conversion relationship between the moving distance of the moving trajectory in the horizontal direction and the yaw angle of the ball model, and determining the target yaw angle of the ball model based on the moving distance of the moving trajectory in the horizontal direction and the conversion relationship.

[0048] Determining the target pitch angle of the ball model based on the moving distance of the moving trajectory in the vertical direction, for example, may include: pre-constructing a conversion relationship between the moving distance of the moving trajectory in the vertical direction and the pitch angle of the ball model, and determining the target pitch angle of the ball model based on the moving distance of the moving trajectory in the vertical direction and the conversion relationship.

[0049] Exemplarily, a right-handed Cartesian coordinate system is constructed with the user's head as the center. The constructed right-handed Cartesian coordinate system is as follows: Figure 4 As shown. Defined in the plane where the video playback window is located, when the horizontal moving distance is equal to the width of the video playback window, the ball model rotates 2π / 3 around the y-axis, that is, 0.66π. When the vertical moving distance is equal to the height of the video playback window, the ball model rotates π / 2 around the x-axis, that is, 0.5π. Based on this, the conversion relationship between the horizontal moving distance d1 of the preset moving trajectory and the yaw angle yaw of the ball model can be expressed as formula (1)

[0050]

[0051] The conversion relationship between the moving distance d2 of the moving trajectory in the vertical direction and the pitch angle pitch of the ball model can be expressed as formula (2):

[0052]

[0053] Among them, s1 is the width of the video playback window, and s2 is the height of the video playback window.

[0054] The target yaw angle and target pitch angle calculated by equation (1) and equation (2) are in radians. The target yaw angle and target pitch angle in radians can be converted into degrees later as needed.

[0055] It should be emphasized that in the above scheme, the roll angle of the ball model is not considered when determining the target pose information of the ball model. The purpose of this setting is to ensure the perception of the VR video image moving horizontally or vertically.

[0056] After determining the target yaw angle of the ball model based on the moving distance of the moving trajectory in the horizontal direction, the method further includes: if the target yaw angle of the ball model is outside the preset yaw angle range, updating the target yaw angle of the ball model, and the updated target yaw angle of the ball model is equal to the preset yaw angle; and / or, after determining the target pitch angle of the ball model based on the moving distance of the moving trajectory in the vertical direction, the method further includes: if the target pitch angle of the ball model is outside the preset pitch angle range; updating the target pitch angle of the ball model, and the updated target pitch angle of the ball model is equal to the preset yaw angle.

[0057] The preset yaw angle range is a pre-specified range, which is used to determine whether rotating the ball model at the target yaw angle will cause the image displayed in the video playback window to move excessively, making it impossible to watch the VR video normally. For example, for a 180° VR video, the video image is only attached to one side of the hemispherical model, and there is no image attached to the other side. If the target yaw angle is too large, causing the ball model to rotate excessively, the VR video image originally in the video playback window will be rotated out of the window, and the surface of the ball model without an image attached will face the video playback window, resulting in no VR video image in the video playback window.

[0058] If the target yaw angle of the ball model is within the preset yaw angle range, it means that if the ball model is rotated at the target yaw angle, the image displayed in the video playback window will not move excessively. If the target yaw angle of the ball model is outside the preset yaw angle range, it means that if the ball model is rotated at the target yaw angle, the image displayed in the video playback window will move excessively, so the preset yaw angle is used as the target yaw angle.

[0059] In practice, the present application does not limit the specific value of the preset yaw angle. When the ball model is rotated at the preset yaw angle, the VR video image will not be moved out of the edge of the video playback window, and the image displayed in the video playback window will not be excessively moved.

[0060] Assume that in a certain scene, a VR video image is attached to the surface of a ball model. When the left edge of the VR video and the left edge of the video playback window are in the same vertical position, the yaw angle of the ball model is the maximum yaw angle (hereinafter referred to as the first angle); and when the right edge of the VR video and the right edge of the video playback window are in the same vertical position, the yaw angle of the ball model is the minimum yaw angle (hereinafter referred to as the second angle). The preset yaw angle range is a range greater than or equal to the second angle and less than or equal to the first angle. Furthermore, it can be set that if the target yaw angle of the ball model is within the preset yaw angle range, the target yaw angle of the ball model remains unchanged. If the target yaw angle of the ball model is outside the preset yaw angle range and is greater than the first angle; update the target yaw angle of the ball model to the first angle; if the target yaw angle of the ball model is outside the preset yaw angle range and is less than the second angle; update the target yaw angle of the ball model to the second angle.

[0061] Similarly, the preset pitch angle range is a pre-specified range, which is used to determine whether rotating the ball model at the target pitch angle will cause the image displayed in the video playback window to flip, such as presenting an upside-down state. If the target pitch angle of the ball model is within the preset pitch angle range, it means that rotating the ball model at the target pitch angle will not cause the image displayed in the video playback window to flip. If the target pitch angle of the ball model is outside the preset pitch angle range, it means that rotating the ball model at the target pitch angle will cause the image displayed in the video playback window to flip.

[0062] In practice, the preset pitch angle has no specific value, and this application does not impose any restrictions on this. When the ball model is rotated at the preset pitch angle, the picture displayed in the video playback window will not flip.

[0063] Assume that in a certain scene, a VR video image is attached to the surface of a ball model. When the upper edge of the VR video is level with the upper edge of the video playback window, the pitch angle of the ball model is the maximum pitch angle (hereinafter referred to as the third angle); and when the lower edge of the VR video is level with the lower edge of the video playback window, the pitch angle of the ball model is the minimum pitch angle (hereinafter referred to as the fourth angle). The preset pitch angle range is a range greater than or equal to the fourth angle and less than or equal to the third angle. Furthermore, it can be set that if the target pitch angle of the ball model is within the preset pitch angle range, the target pitch angle of the ball model remains unchanged. If the target pitch angle of the ball model is outside the preset pitch angle range and is greater than the third angle; update the target pitch angle of the ball model to the third angle; if the target pitch angle of the ball model is outside the preset pitch angle range and is less than the fourth angle; update the target pitch angle of the ball model to the fourth angle.

[0064] It should also be noted that, in practice, for 360° VR videos, the target yaw angle in the target posture information of the ball model may not be limited to be within the preset yaw angle range, and the target pitch angle may only be limited to be within the preset pitch angle range. However, for 180° VR videos, the target yaw angle in the target posture information of the ball model is limited to be within the preset yaw angle range, and the target pitch angle is limited to be within the preset pitch angle range.

[0065] In other embodiments, optionally, the adjustment operation on the VR video screen includes a scaling operation on the VR video screen, and in response to the adjustment operation on the VR video screen, the target posture information of the ball model is determined, including: determining the target scaling ratio of the VR video screen; based on the target scaling ratio of the VR video screen, determining the target distance between the center of the ball model and the video playback window; determining that the target posture information of the ball model includes the target distance.

[0066] The scaling operation on the VR video screen may, for example, indicate that the user wishes to zoom in or out on the VR video screen. It should be noted that in the process of zooming in or out on the VR video screen, the relative position relationship between the video playback window and the user in the extended reality space will not be changed, nor will the size of the video playback window be changed.

[0067] The target scaling ratio of the VR video screen can be determined based on the scaling operation of the VR video screen. For example, if multiple options for scaling the VR video screen are displayed in the extended space, each option corresponds to a scaling ratio of the VR video screen, and different options correspond to different scaling ratios of the VR video screen. The scaling ratio of the VR video screen corresponding to the option selected by the user is used as the target scaling ratio.

[0068] In practice, a correspondence between the target scaling ratio of the VR video screen and the distance between the center of the ball model and the video playback window can be pre-constructed, and "determining the target distance between the center of the ball model and the video playback window based on the target scaling ratio of the VR video screen" may include: determining the target distance between the center of the ball model and the video playback window based on the target scaling ratio of the VR video screen and the correspondence.

[0069] Specifically, when the VR video screen needs to be enlarged, the distance between the center of the ball model and the video playback window is reduced. When the VR video screen needs to be reduced, the distance between the center of the ball model and the video playback window is increased.

[0070] For example, see Figure 5, EF is the video playback window. In the initial state, the user is at point D1, and the user watches the VR video through the video playback window EF. The VR video screen displayed by the video playback window EF is obtained based on the ball model located at L1. At this time, point D1 is also the center of L1. Point D1 is located on the perpendicular bisector of the video playback window EF. Angle γ is the user's maximum horizontal field of view limited by the video playback window. At a certain moment, the user zooms in on the VR video screen, moves the center of the ball model from point D1 to point D2, and the distance between the center of the ball model and the video playback window decreases. As the center of the ball model moves from point D1 to point D2, the ball model moves from L1 to L2. Obviously, while keeping the relative position of the user and the video playback window in the extended reality space unchanged, the user's maximum horizontal field of view γ remains unchanged due to the limitation of the video playback window. When the ball model moves from L1 to L2, the user changes from seeing the image of the H1H2 area to seeing the image of the J1J2 area. The picture seen by the user becomes farther and the content becomes more, achieving the purpose of zooming in on the VR video screen. At another moment, the user zooms out of the VR video screen, moves the center of the ball model from point D1 to point D3, and the distance between the center of the ball model and the video playback window increases. As the center of the ball model moves from point D1 to point D3, the ball model moves from L1 to L3. Obviously, while keeping the relative position of the user and the video playback window in the extended reality space unchanged, the user's maximum horizontal field of view γ remains unchanged due to the limitation of the video playback window. When the ball model moves from L1 to L3, the user changes from seeing the image of the H1H2 area to seeing the image of the G1G2 area. The picture the user sees becomes closer and the content becomes smaller, achieving the purpose of reducing the VR video screen.

[0071] The above technical solution is to display a video playback window in the extended reality space; the video playback window is used to display the image content of a local area in the VR video image; in response to the adjustment operation of the VR video screen, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is adjusted. The screen displayed in the video playback window after the adjustment is not completely the same as the screen content displayed in the video playback window before the adjustment. Its essence is to provide a method that allows users to adjust the viewing angle of VR videos according to their own needs. It can meet the personalized viewing needs of users.

[0072] On the basis of the above technical solution, optionally, in response to a reset operation on the VR video screen, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is reset to the local area displayed in the video playback window at an initial moment, and the initial moment is the moment before the adjustment operation on the VR video screen is detected.

[0073] The reset operation of the VR video screen may be, for example, an operation expressing that the user wishes to restore the screen displayed in the video playback window to the state before adjustment. Specifically, it may be an operation of clicking a reset option in the extended reality space, or performing a preset gesture operation, or a language instruction issued by the user for resetting the video screen.

[0074] In practice, after the user makes a series of adjustments to the VR video screen, it may happen that the VR video screen deviates increasingly from the optimal viewing position or angle. By responding to the reset operation of the VR video screen, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is reset to the local area displayed in the video playback window at the initial moment. This allows the user to watch the video again from a preset and more reasonable perspective, thereby avoiding missing important content or affecting the viewing experience due to improper operation.

[0075] Furthermore, the extended reality space also includes a playback control option, and the method may also include: the playback control option is used to control the state of the VR video played in the video playback window; the method also includes: in response to an adjustment operation on the VR video screen, stopping displaying the playback control option in the extended reality space.

[0076] The playback control option may be, for example, a functional option displayed in the extended reality space for playing or controlling the video. Exemplarily, the playback control option may be, for example, at least one of the following: a play / pause control, a volume adjustment control, a fast forward control, a fast rewind control, a viewing mode selection control, a stereo effect selection control, a panoramic angle selection control, a subtitle switch control, an audio track selection control, a spatial audio control, a playlist control, a video 2D / 3D type selection control, a VR projection format selection control, and a viewing scene selection control.

[0077] Setting the display of the broadcast control options in the extended reality space to stop in response to the adjustment operation of the VR video screen can help avoid accidentally touching the broadcast control options during the process of adjusting the VR video screen.

[0078] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0079] Figure 6 Schematic diagram of the structure of an interactive device in an embodiment of the present disclosure. The interactive device provided in an embodiment of the present disclosure can be configured in an extended reality device. Figure 6 , the interactive device specifically includes:

[0080] A display module 310 is used to display a video playback window in an extended reality space; the video playback window is used to display image content of a local area in a VR video image;

[0081] The adjustment module 320 is used to respond to the adjustment operation of the VR video picture, keep the position of the video playback window fixed in the extended reality space, and adjust the local area of ​​the VR video image displayed in the video playback window. The picture displayed in the video playback window after the adjustment is not completely the same as the picture content displayed in the video playback window before the adjustment.

[0082] Furthermore, the VR video has a corresponding ball model, and the adjustment module 320 is used to:

[0083] In response to the adjustment operation of the VR video picture, the target position information of the ball model is determined; the target position information of the ball model includes the target position information and / or the target posture information of the ball model in the extended reality space;

[0084] Based on the target posture information of the ball model, the position and / or posture of the ball model in the extended reality space is adjusted.

[0085] Furthermore, the adjustment operation on the VR video screen includes a movement operation on the VR video screen, and the adjustment module 320 is used to:

[0086] Based on the movement trajectory of the VR video screen, determine the movement distance of the movement trajectory in the horizontal direction and the movement distance in the vertical direction;

[0087] Determining a target yaw angle of the ball model based on a moving distance of the moving trajectory in a horizontal direction;

[0088] Determining a target pitch angle of the ball model based on a moving distance of the moving trajectory in a vertical direction;

[0089] Determine the target posture information of the ball model including the target yaw angle and target pitch angle of the ball model.

[0090] Furthermore, the adjustment module 320 is used to:

[0091] After determining the target yaw angle of the ball model based on the moving distance of the moving trajectory in the horizontal direction, the method further includes: if the target yaw angle of the ball model is outside the preset yaw angle range, updating the target yaw angle of the ball model, and the updated target yaw angle of the ball model is equal to the preset yaw angle; and / or,

[0092] After determining the target pitch angle of the ball model based on the moving distance of the moving trajectory in the vertical direction, the method further includes: if the target pitch angle of the ball model is outside a preset pitch angle range; updating the target pitch angle of the ball model, and after the update, the target pitch angle of the ball model is equal to a preset yaw angle.

[0093] Furthermore, the adjustment module 320 is used to:

[0094] Determining a target scaling ratio of the VR video image;

[0095] Determining a target distance between the center of the ball model and the video playback window based on a target scaling ratio of the VR video screen;

[0096] Determine the target pose information of the ball model including the target distance.

[0097] Furthermore, the device also includes a reset module, which is used to:

[0098] In response to a reset operation on the VR video screen, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is reset to the local area displayed in the video playback window at an initial moment, and the initial moment is the moment before the adjustment operation on the VR video screen is detected.

[0099] Furthermore, the extended reality space further includes a play control option, and the play control option is used to control the state of the VR video played in the video play window; the device also includes an option display state adjustment module, which is used to:

[0100] In response to an adjustment operation on the VR video screen, the broadcast control option is stopped from being displayed in the extended reality space.

[0101] The interactive device provided in the embodiment of the present disclosure can execute the steps included in the interactive method provided in the embodiment of the method of the present disclosure, and has the execution steps and beneficial effects, which will not be repeated here.

[0102] Figure 7 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 7 , which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 7The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0103] like Figure 7 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003 to implement the interactive method of the embodiment described in the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0104] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although Figure 7 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0105] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the interactive method as described above. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0106] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include an information signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated information signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0107] In some embodiments, the client and the server may communicate using any known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any known or future developed network.

[0108] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0109] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0110] Displaying a video playback window in the extended reality space; the video playback window is used to display the image content of a local area in the VR video image;

[0111] In response to the adjustment operation of the VR video picture, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is adjusted. The picture displayed in the video playback window after the adjustment is not completely the same as the picture content displayed in the video playback window before the adjustment.

[0112] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0116] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0119] one or more processors;

[0120] A memory for storing one or more programs;

[0121] When the one or more programs are executed by the one or more processors, the one or more processors implement any interaction method provided in the present disclosure.

[0122] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the program implements any of the interaction methods provided by the present disclosure.

[0123] The embodiments of the present disclosure further provide a computer program product, which includes a computer program or instructions, and the computer program or instructions implement the above-mentioned interaction method when executed by a processor.

[0124] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0125] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interactive method, characterized in that: include: Display the video playback window in the extended reality space; The video playback window is used to display the image content of a local area in the VR video image; In response to the adjustment operation of the VR video picture, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is adjusted. The picture displayed in the video playback window after the adjustment is not completely the same as the picture content displayed in the video playback window before the adjustment.

2. The method according to claim 1, characterized in that The VR video has a corresponding ball model, and in response to the adjustment operation on the VR video screen, the local area of ​​the VR video image displayed in the video playback window is adjusted, including: In response to the adjustment operation of the VR video picture, the target position information of the ball model is determined; the target position information of the ball model includes the target position information and / or the target posture information of the ball model in the extended reality space; Based on the target posture information of the ball model, the position and / or posture of the ball model in the extended reality space is adjusted.

3. The method according to claim 2, characterized in that The adjustment operation on the VR video screen includes a movement operation on the VR video screen, and the determination of the target position information of the ball model in response to the adjustment operation on the VR video screen includes: Based on the movement trajectory of the VR video screen, determine the movement distance of the movement trajectory in the horizontal direction and the movement distance in the vertical direction; Determining a target yaw angle of the ball model based on a moving distance of the moving trajectory in a horizontal direction; Determining a target pitch angle of the ball model based on a moving distance of the moving trajectory in a vertical direction; Determine the target posture information of the ball model including the target yaw angle and target pitch angle of the ball model.

4. The method according to claim 3, characterized in that: After determining the target yaw angle of the ball model based on the moving distance of the moving trajectory in the horizontal direction, the method further includes: if the target yaw angle of the ball model is outside the preset yaw angle range, updating the target yaw angle of the ball model, and the updated target yaw angle of the ball model is equal to the preset yaw angle; and / or, After determining the target pitch angle of the ball model based on the moving distance of the moving trajectory in the vertical direction, the method further includes: if the target pitch angle of the ball model is outside a preset pitch angle range; updating the target pitch angle of the ball model, and after the update, the target pitch angle of the ball model is equal to a preset yaw angle.

5. The method according to claim 2, characterized in that: The adjusting operation on the VR video screen includes a scaling operation on the VR video screen, and the determining the target position information of the ball model in response to the adjusting operation on the VR video screen includes: Determining a target scaling ratio of the VR video image; Determining a target distance between the center of the ball model and the video playback window based on a target scaling ratio of the VR video screen; Determine the target pose information of the ball model including the target distance.

6. The method according to claim 2, characterized in that Also includes: In response to a reset operation on the VR video screen, the position of the video playback window in the extended reality space is kept fixed, and the local area of ​​the VR video image displayed in the video playback window is reset to the local area displayed in the video playback window at an initial moment, and the initial moment is the moment before the adjustment operation on the VR video screen is detected.

7. The method according to claim 1, characterized in that The extended reality space further includes a play control option, and the play control option is used to control the state of the VR video played in the video play window; the method further includes: In response to an adjustment operation on the VR video screen, the broadcast control option is stopped from being displayed in the extended reality space.

8. An interactive device, characterized in that: include: A display module, used to display a video playback window in an extended reality space; The video playback window is used to display the image content of a local area in the VR video image; The adjustment module is used to respond to the adjustment operation of the VR video picture, keep the position of the video playback window fixed in the extended reality space, and adjust the local area of ​​the VR video image displayed in the video playback window. The picture displayed in the video playback window after the adjustment is not completely the same as the picture content displayed in the video playback window before the adjustment.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images