Interaction method and device, electronic equipment and storage medium

Abstract

The invention relates to an interaction method and device, electronic equipment and a storage medium. The method comprises the following steps: displaying a broadcast control component and a relative pose locking option in an augmented reality space; the playing control component comprises a video playing window and a playing control option, and the video playing window is used for playing a video; in response to a selection operation on the relative pose locking option, setting the relative pose of the head of the user and the broadcast control component to be in a locking state; under the condition that the relative pose of the head of the user and the broadcast control assembly is in the locked state, the broadcast control assembly moves along with the head of the user, and the relative pose of the head of the user and the broadcast control assembly is kept unchanged all the time. By adopting the technical scheme provided by the invention, even if the head position and posture of the user are changed, the head angle does not need to be frequently adjusted to keep the video playing window in the visual field range, so that the fatigue feeling of the user when watching the video is relieved. In addition, the immersion state of the video watched by the user cannot be interrupted due to the change of the head position and posture.

Description

Technical Field

[0001] The present disclosure relates to the technical field of extended reality, and in particular to an interaction method, device, electronic device and storage medium. Background Art

[0002] With the continuous advancement of technology, extended reality (XR) devices have gradually entered people's lives, and their functions have become increasingly rich. Among them, the video viewing function has attracted much attention from users. At present, most XR devices mainly use the video playback window in the XR environment constructed by the device to achieve video playback. However, in actual applications, due to the unreasonable setting of the existing video playback window, it is easy to cause viewing fatigue to users and affect the immersive feeling of watching videos. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an interaction method, an apparatus, an electronic device and a storage medium.

[0004] In a first aspect, the present disclosure provides an interaction method, comprising:

[0005] Displaying a playback control component and a relative posture locking option in an extended reality space; the playback control component includes a video playback window and playback control options, and the video playback window is used to play the video;

[0006] In response to a selection operation on the relative posture locking option, the relative posture between the user's head and the broadcast control component is set in a locked state; when the relative posture between the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture between the user's head and the broadcast control component always remains unchanged.

[0007] In a second aspect, the present disclosure further provides an interactive device, including:

[0008] A display module, used to display a playback control component and a relative posture locking option in an extended reality space; the playback control component includes a video playback window and playback control options, and the video playback window is used to play a video;

[0009] A setting module is used to set the relative posture of the user's head and the broadcast control component to be in a locked state in response to a selection operation on the relative posture locking option; when the relative posture of the user's head and the broadcast control component are in a locked state, the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged.

[0010] In a third aspect, the present disclosure further provides an electronic device, the electronic device comprising:

[0011] one or more processors;

[0012] A storage device for storing one or more programs;

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method as described above.

[0014] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned interaction method when executed by a processor.

[0015] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0016] The technical solution provided by the embodiment of the present disclosure is to display the broadcast control component and the relative posture locking option in the extended reality space; the broadcast control component includes a video playback window and a playback control option, and the video playback window is used to play the video; in response to the selection operation of the relative posture locking option, the relative posture of the user's head and the broadcast control component is set to be in a locked state; when the relative posture of the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component remains unchanged. Its essence is to provide a video viewing mode, in which the broadcast control component can move with the user's head, and the relative posture of the two is always constant. In this way, even if the user's head position and posture change, there is no need to frequently adjust the head angle to keep the video playback window within the field of view, thereby reducing the user's fatigue when watching the video. In addition, the user's immersive state of watching the video will not be interrupted by changes in head position and posture. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of an interactive method provided by an embodiment of the present disclosure;

[0020] Figure 2 A schematic diagram of a panoramic video playback method provided by an embodiment of the present disclosure;

[0021] Figure 3 is a schematic diagram of the structure of an interactive device in an embodiment of the present disclosure;

[0022] Figure 4 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0025] As described in the background technology, the existing video playback window setting scheme has defects, which can easily cause users to experience viewing fatigue and damage the immersive experience of watching videos. After in-depth analysis, the applicant believes that one cause of this problem is that when the user is in a mobile state, the video playback window is always in a fixed position in the extended reality space. Specifically, because the video playback window is fixed in the extended reality space, when the user moves, if he needs to continue watching the video, he needs to constantly adjust the head angle to keep the video playback window in the field of vision. This frequent head movement can easily cause users to feel tired. In addition, since the video playback window is in a constant position in the extended reality space, once the user starts to move, the immersive state of watching the video will be broken.

[0026] In view of this, Figure 1 This is a flowchart of an interaction method provided in an embodiment of the present disclosure. This embodiment is applicable to situations where interaction is performed in a client. The method can be executed by an interaction device, which can be implemented in software and / or hardware, and can be configured in an extended reality device.

[0027] The extended reality devices described in this application may include but are not limited to the following types:

[0028] The computer-side extended reality device uses the PC to perform related calculations and data output of the extended reality function, and the external computer-side extended reality device uses the data output by the PC to achieve the extended reality effect.

[0029] Mobile extended reality devices support the setting of mobile terminals (such as smart phones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations of the extended reality function and outputs data to the mobile extended reality device, such as watching extended reality videos through the mobile terminal's APP.

[0030] The all-in-one extended reality device has a processor for performing relevant calculations of virtual functions, and thus has independent extended reality input and output functions. It does not need to be connected to a PC or mobile terminal and has a high degree of freedom in use.

[0031] like Figure 1 As shown, the method may specifically include:

[0032] S110. Displaying a playback control component and a relative posture locking option in an extended reality space; the playback control component includes a video playback window and playback control options, and the video playback window is used to play the video.

[0033] The extended reality space can be, for example, a virtual scene created with the help of an environmental image when the extended reality device is running. The virtual scene can be a simulated environment of the real world, a semi-simulated and semi-fictional virtual scene, or a purely fictional virtual scene. The extended reality space can include the sky, land, ocean, etc. The land can include environmental elements such as deserts and cities, and users can move in the extended reality space. The environmental image is equivalent to the desktop wallpaper displayed after the computer is turned on.

[0034] The playback control component may be, for example, a functional component displayed in the extended reality space for playing and controlling the video. In practice, the playback control component may be displayed in the extended reality space in the form of a panel.

[0035] The video playback window may be, for example, an area of ​​the playback control component, which is used to play the video. The video played in the video playback window may include at least one of the following: ordinary (non-VR) 2D video, ordinary (non-VR) 3D video, panoramic video, 2D fisheye video, 3D fisheye video, and spatial video.

[0036] The playback control options may, for example, be at least one of the following: a play / pause control, a volume adjustment control, a fast-forward control, a fast-rewind control, a viewing mode selection control, a stereoscopic effect selection control, a panoramic angle selection control, a subtitle switch control, an audio track selection control, a spatial audio control, a playlist control, a video 2D / 3D type selection control, a VR projection format selection control, and a viewing scene selection control.

[0037] In practice, the video playback window and the playback control options may be located in different spatial planes of the extended reality space, or the video playback window and the playback control options may be located in different position areas of the same plane in the extended reality space, and the embodiments of the present disclosure do not limit this. For example, the position and orientation of the video playback window and the playback control options in the extended reality space may be set based on the viewing distance of the user, for example, the distance between the video playback window and the user is farther than the distance between the playback control options and the user.

[0038] The relative posture lock option may be, for example, an option for setting whether to enable the relative posture lock function. The relative posture lock option may be an option for controlling the playback mode, independent of other playback control options; it may also be a subordinate option of a playback control option; it may also be a subordinate option of system settings. System settings may be, for example, a set of functions for configuring and managing the overall operating environment, functional characteristics, and user personalized preferences of a device or software.

[0039] The relative pose lock option and the playback control option can be located in different position areas of the same plane in the extended reality space, or in different spatial planes of the extended reality space.

[0040] S120. In response to a selection operation on a relative posture lock option, the relative posture of the user's head and the broadcast control component is set to a locked state; when the relative posture of the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged.

[0041] The selection operation of the relative posture lock option may be, for example, an operation indicating that the user wishes to enable the relative posture lock function. Exemplarily, the selection operation of the relative posture lock option may be, for example, a click operation or a drag operation on the relative posture lock option.

[0042] The relative position and posture of the user's head and the broadcast control component may be, for example, the relative position and relative posture of the user's head and the broadcast control component in the extended reality space. The relative position reflects the distance relationship between the two in the three-dimensional space, and the relative posture reflects the orientation relationship between the two.

[0043] When the relative position of the user's head and the broadcast control component is in a locked state, no matter how the user's head moves or what angle it rotates, the relative position and orientation between it and the broadcast control component maintain the set initial state. From the user experience perspective, when the relative position of the user's head and the broadcast control component is in a locked state, the broadcast control component seems to be glued to a fixed distance and angle in front of the user's eyes. No matter the user's head moves up and down, left and right, forward and backward, or even turns, the broadcast control component always maintains its relative position in the user's field of vision, and there will be no angle tilt or distance change. In this way, no matter how the user changes the position and direction of his head in the virtual space, he can easily watch the video and operate the playback control options through the video playback window, without having to find the position of the broadcast control component again or adjust the viewing angle to adapt to the broadcast control component.

[0044] The above technical solution is provided by displaying the broadcast control component and the relative posture locking option in the extended reality space; the broadcast control component includes a video playback window and a playback control option, and the video playback window is used to play the video; in response to the selection operation of the relative posture locking option, the relative posture of the user's head and the broadcast control component is set to be in a locked state; when the relative posture of the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component remains unchanged. Its essence is to provide a video viewing mode, in which the broadcast control component can move with the user's head, and the relative posture of the two is always constant. In this way, even if the user's head position and posture change, there is no need to frequently adjust the head angle to keep the video playback window within the field of view, thereby reducing the user's fatigue when watching the video. In addition, the user's immersive state of watching the video will not be interrupted by changes in the head position and posture.

[0045] On the basis of the above technical solution, optionally, the method may also include: when the relative posture of the user's head and the broadcast control component is in a locked state, obtaining the user's head posture change information; based on the user's head posture change information, determining the first posture information of the broadcast control component; setting the posture information of the broadcast control component as the first posture information, so that the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged.

[0046] In an extended reality device, a variety of sensors are usually provided, such as an IMU (Inertial Measurement Unit), a 3Dof & 6Dof (three degrees of freedom & six degrees of freedom) sensor, and an eye tracking camera (e.g., an infrared (IR) or near infrared (NIR) camera). With the help of the data collected by these sensors, the user's head posture change information can be determined.

[0047] The first posture information may be, for example, the position and posture to which the broadcast control component needs to be adjusted in order to achieve the purpose of "the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged".

[0048] There are many specific implementation methods for "determining the first position information of the broadcast control component based on the user's head posture change information", and the present application is limited to this. Exemplarily, the specific implementation method of "determining the first position information of the broadcast control component based on the user's head posture change information" may include: determining the initial posture information of the broadcast control component in the extended reality space at the initial moment, and the initial posture information of the user's head in the extended reality space; the initial moment is the moment when the selection operation for the relative posture lock option is detected; based on the initial posture information of the broadcast control component in the extended reality space at the initial moment, and the initial posture information of the user's head in the extended reality space, determine the relative posture between the user's head and the broadcast control component; based on the user's head posture change information and the relative posture between the user's head and the broadcast control component, determine the first position information of the broadcast control component.

[0049] Exemplarily, if the operation of the user clicking the relative posture lock option is regarded as a selection operation of the relative posture lock option, the initial moment is the moment when the user clicks the relative posture lock option. If the posture information includes position information and posture information; the initial position information Pos1 of the video playback window in the extended reality space, the initial position information Pos2 of the playback control option in the extended reality space, and the initial position information PosU of the user's head in the extended reality space are recorded at the initial moment, then the relative position difference △Pos1 between the user's head and the video playback window, and the relative position difference △Pos2 between the user's head and the playback control option, then:

[0050] △Pos1=PosU-Pos1

[0051] △Pos2=PosU-Pos2

[0052] At a certain moment later, the user's head posture changes. Assume that after the user's head posture changes, the new position information of the user's head in the extended reality space is PosU'. Let Pos1' represent the new position information of the video playback window in the extended reality space, and Pos2' represent the new position information of the playback control option in the extended reality space. Then

[0053] Pos1'=PosU'-△Pos1

[0054] Pos2'=PosU'-△Pos2.

[0055] The position information of the video playback window is set to Pos1', and the position information of the playback control option is set to Pos2'. Assuming that after the user's head posture changes, the Euler angle rotated by the user is EulerU, based on the Euler angle rotated by the user EulerU, the target Euler angle of the playback control component can be determined, and then the rotation angle of the playback control component can be adjusted to be equal to the target Euler angle.

[0056] It should be noted that, in practice, when rendering each frame of image, it is necessary to execute the steps of "obtaining user head posture change information when the relative posture of the user head and the broadcast control component is in a locked state; determining the first position information of the broadcast control component based on the user head posture change information; and setting the posture information of the broadcast control component as the first position information".

[0057] On the basis of the above technical solution, optionally, if the video played by the broadcast control component is a panoramic video; the panoramic video has a corresponding geometric model; the relative posture of the broadcast control component and the geometric model is fixed; the method also includes: determining the relative posture of the broadcast control component and the geometric model; determining the second posture information of the geometric model based on the first posture information of the broadcast control component and the relative posture of the broadcast control component and the geometric model; setting the posture information of the geometric model to the second posture information.

[0058] The geometric model corresponding to the panoramic video may be a three-dimensional geometric model for carrying the panoramic video image, and its surface is used as a mapping carrier for the panoramic video image content. There is a correspondence between the points on the surface of the geometric model and the pixels in the panoramic video image, and this correspondence can be determined with the help of a specific projection method. In practice, the geometric model corresponding to the panoramic video may include a cube model or a spherical model, etc. If the geometric model corresponding to the panoramic video is a spherical model, the projection method used to determine the correspondence between the points on the surface of the spherical model and the pixels in the panoramic video image may be a spherical projection method such as equirectangular projection or azimuthal equal-area projection AEP (Adjusted equal-areaprojection). With the correspondence between the points on the surface of the geometric model and the pixels in the panoramic video image, the panoramic image can be accurately "attached" to the surface of the geometric model.

[0059] In some scenarios, a panoramic video is played using a video playback window. The video playback window is similar to a window. Users watch the panoramic video through the video playback window, which is similar to users observing the scenery outside the window "through" the window. For details, see Figure 2 , the extended reality space includes the user ( Figure 2 D1 represents the user's position in the extended reality space), video playback windows A1A2A3A4 and a ball model corresponding to the panoramic video. Figure 2 In the figure, ring L1 and ring L2 are two rings on the ball model corresponding to the panoramic video. The user at position D1 can watch the panoramic video image "attached" to the ball model through the video playback window A1A2A3A4. And, due to the limitation of the video playback window A1A2A3A4, the user can only see the image of the local area "attached" to the surface of the ball model. For example, when the user is in Figure 2 When the position D1 is in the middle, only the image of the area B1B2B3B4 "attached" to the surface of the ball model can be seen. In other words, when the panoramic video is played using the video playback window, the video playback window can only display the image of a specific area of ​​the panoramic video image.

[0060] The second posture information of the geometric model may be, for example, the position and posture to which the geometric model needs to be adjusted in order to achieve the purpose of "the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged".

[0061] Normally, when a user selects the relative posture lock option, it indicates that the user expects the video playback window to keep displaying the image of the specific area in the panoramic video image. The second posture information of the geometric model is determined by setting the first posture information based on the broadcast control component and the relative posture of the broadcast control component and the geometric model; the posture information of the geometric model is set to the second posture information, so that when the relative posture of the user's head and the broadcast control component is locked, even if the user's head position and / or posture changes, the video playback window always displays the image of the same specific area of ​​the panoramic video image. In this way, when the user is watching the panoramic video, no matter how his head moves, he will not miss the image details in the specific area.

[0062] Based on the above technical solution, optionally, the method may also include: when the extended reality device is in perspective mode, determining the user's head movement distance based on the user's head posture change information; if the user's head movement distance is greater than a set distance threshold, increasing the transparency of the broadcast control component.

[0063] Perspective mode, for example, means that in the extended reality space, in addition to the broadcast control component, the user can also observe the real external environment. In this case, it can be achieved by capturing images of the real external environment where the user is through sensors such as cameras, and superimposing the images of the broadcast control component on these images, thereby forming a perspective mode visual effect.

[0064] Increasing the transparency of the broadcast control component can reveal things that are blocked by the broadcast control component. In the perspective mode, the things blocked by the broadcast control component are things in the real external environment where the user is located. By setting the extended reality device in the perspective mode, the user's head movement distance is determined based on the user's head posture change information; if the user's head movement distance is greater than the set distance threshold, the transparency of the broadcast control component is increased, which can avoid the safety risks caused by the broadcast control component blocking the user's field of view during the user's movement. It can meet the needs of users, such as watching videos while moving in a small range indoors.

[0065] On the basis of the above technical solution, optionally, a relative posture locking option is displayed in the extended reality space, including: when the extended reality device is in a first working mode, a relative posture locking option is displayed in the extended reality space; the first working mode includes a completely black mode or a perspective mode.

[0066] The full black mode may be, for example, an interactive model in which, in the extended reality space, the user cannot observe other virtual or real objects except the broadcast control component. In other words, in the full black mode, except for the broadcast control component, the rest of the space appears black.

[0067] In the all-black mode, external interference is completely blocked. If the user sets the working mode of the extended reality device to the all-black mode, it means that the user wants to watch the movie immersively. In this case, by setting the relative posture lock option displayed in the extended reality space, it means allowing the user's head to be locked in relative posture with the broadcast control component, so as to further reduce the number of times the user's immersive state of watching the video is interrupted, which helps to improve the user's immersive experience.

[0068] If the user sets the working mode of the extended reality device to perspective mode, it means that the user needs to see the real environment around him. In this case, by setting the relative posture lock option displayed in the extended reality space, it means allowing the user's head to be locked in relative posture with the playback control component, so that the user does not need to frequently adjust the head angle to keep the video playback window within the field of view, thereby reducing the user's fatigue when watching videos and improving the user's viewing experience.

[0069] Furthermore, in response to an instruction to switch the working mode of the extended reality device to a second working mode, the relative position of the user's head and the broadcast control component is set to an unlocked state; when the relative position of the user's head and the broadcast control component is in an unlocked state, the broadcast control component does not move with the user's head, and the relative position of the user's head and the broadcast control component is variable; the second working mode includes a virtual viewing mode.

[0070] The virtual viewing mode can be, for example, in an extended reality space, where in addition to the broadcast control components, the user can also observe things in the virtual environment. In this viewing mode, the user is completely isolated from the external environment, and the virtual environment he sees is purely fictional.

[0071] The original intention of the design of the virtual viewing mode is to simulate the user watching a video in a virtual cinema or other viewing environment. In this scenario, the user can freely observe various directions and details in the virtual environment, such as looking up at the ceiling of the virtual cinema, or turning his head to observe the virtual audience around him, so that the user can be more deeply immersed in the virtual viewing atmosphere. By setting a response to the instruction to switch the working mode of the extended reality device to the second working mode, setting the relative posture of the user's head and the broadcast control component to be in an unlocked state, it is conducive to allowing the user to freely observe various directions and details in the virtual environment.

[0072] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0073] Figure 3 Schematic diagram of the structure of an interactive device in an embodiment of the present disclosure. The interactive device provided in the embodiment of the present disclosure can be configured in a client or in a server. Figure 3 , the interactive device specifically includes:

[0074] The display module 310 is used to display the playback control component and the relative posture locking option in the extended reality space; the playback control component includes a video playback window and playback control options, and the video playback window is used to play the video;

[0075] The setting module 320 is used to set the relative posture of the user's head and the broadcast control component to be in a locked state in response to the selection operation of the relative posture locking option; when the relative posture of the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged.

[0076] Furthermore, the device also includes an adjustment module, which is used to:

[0077] When the relative posture of the user's head and the broadcast control component is in a locked state, obtaining the user's head posture change information;

[0078] Based on the user's head posture change information, determining the first position information of the broadcast control component;

[0079] The posture information of the broadcast control component is set to the first posture information, so that the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged.

[0080] Furthermore, the adjustment module is used to:

[0081] Determine the initial posture information of the broadcast control component in the extended reality space and the initial posture information of the user's head in the extended reality space at an initial moment; the initial moment is the moment when a selection operation is detected for the relative posture locking option;

[0082] Determine the relative posture between the user's head and the broadcast control component based on the initial posture information of the broadcast control component in the extended reality space at the initial moment and the initial posture information of the user's head in the extended reality space;

[0083] Based on the user's head posture change information and the relative posture between the user's head and the broadcast control component, the first posture information of the broadcast control component is determined.

[0084] Furthermore, if the video played by the broadcast control component is a panoramic video; the panoramic video has a geometric model corresponding thereto; the relative position of the broadcast control component and the geometric model is fixed; the adjustment module is used to:

[0085] Determining the relative position and posture of the broadcast control component and the geometric model;

[0086] Determine second pose information of the geometric model based on the first pose information of the broadcast control component and the relative pose of the broadcast control component and the geometric model;

[0087] The pose information of the geometric model is set as second pose information.

[0088] Furthermore, the adjustment module is used to:

[0089] When the extended reality device is in perspective mode, determining a movement distance of the user's head based on the user's head posture change information;

[0090] If the user's head moves a distance greater than a set distance threshold, the transparency of the broadcast control component is increased.

[0091] Furthermore, the display module is used to:

[0092] When the extended reality device is in a first working mode, a relative posture locking option is displayed in the extended reality space; the first working mode includes a completely black mode or a perspective mode.

[0093] Furthermore, the device also includes an exit module, which is used to:

[0094] In response to an instruction to switch the working mode of the extended reality device to a second working mode, the relative position of the user's head and the broadcast control component is set to be in an unlocked state; when the relative position of the user's head and the broadcast control component is in an unlocked state, the broadcast control component does not move with the user's head, and the relative position of the user's head and the broadcast control component is variable; the second working mode includes a virtual viewing mode.

[0095] The interactive device provided in the embodiment of the present disclosure can execute the steps executed by the client or server in the interactive method provided in the embodiment of the method of the present disclosure, and has execution steps and beneficial effects, which will not be repeated here.

[0096] Figure 4 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 4 , which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0097] like Figure 4 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003 to implement the interactive method of the embodiment described in the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0098] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although Figure 4 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0099] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the interactive method as described above. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0100] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include an information signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated information signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0101] In some embodiments, the client and the server may communicate using any known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any known or future developed network.

[0102] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0103] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0104] Displaying a playback control component and a relative posture locking option in an extended reality space; the playback control component includes a video playback window and playback control options, and the video playback window is used to play the video;

[0105] In response to a selection operation on the relative posture locking option, the relative posture between the user's head and the broadcast control component is set in a locked state; when the relative posture between the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture between the user's head and the broadcast control component always remains unchanged.

[0106] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0107] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0110] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0111] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0112] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0113] one or more processors;

[0114] A memory for storing one or more programs;

[0115] When the one or more programs are executed by the one or more processors, the one or more processors implement any interaction method provided in the present disclosure.

[0116] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the program implements any of the interaction methods provided by the present disclosure.

[0117] The embodiments of the present disclosure further provide a computer program product, which includes a computer program or instructions, and the computer program or instructions implement the above-mentioned interaction method when executed by a processor.

[0118] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0119] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interactive method, characterized in that: include: Display the broadcast control components and relative pose locking options in the extended reality space; The playback control component includes a video playback window and playback control options, and the video playback window is used to play the video; In response to a selection operation on the relative posture locking option, the relative posture between the user's head and the broadcast control component is set in a locked state; when the relative posture between the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture between the user's head and the broadcast control component always remains unchanged.

2. The method according to claim 1, characterized in that Also includes: When the relative posture of the user's head and the broadcast control component is in a locked state, obtaining the user's head posture change information; Based on the user's head posture change information, determining the first position information of the broadcast control component; The posture information of the broadcast control component is set to the first posture information, so that the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged.

3. The method according to claim 2, characterized in that The determining the first position information of the broadcast control component based on the user's head position change information includes: Determine the initial posture information of the broadcast control component in the extended reality space and the initial posture information of the user's head in the extended reality space at an initial moment; the initial moment is the moment when a selection operation is detected for the relative posture locking option; Determine the relative posture between the user's head and the broadcast control component based on the initial posture information of the broadcast control component in the extended reality space at the initial moment and the initial posture information of the user's head in the extended reality space; Based on the user's head posture change information and the relative posture between the user's head and the broadcast control component, the first posture information of the broadcast control component is determined.

4. The method according to claim 2, characterized in that: If the video played by the broadcast control component is a panoramic video; the panoramic video has a geometric model corresponding to it; the relative position of the broadcast control component and the geometric model is fixed; the method further includes: Determining the relative position and posture of the broadcast control component and the geometric model; Determine second pose information of the geometric model based on the first pose information of the broadcast control component and the relative pose of the broadcast control component and the geometric model; The pose information of the geometric model is set as second pose information.

5. The method according to claim 2, characterized in that: Also includes: When the extended reality device is in perspective mode, determining a movement distance of the user's head based on the user's head posture change information; If the user's head moves a distance greater than a set distance threshold, the transparency of the broadcast control component is increased.

6. The method according to claim 1, characterized in that The relative pose locking options displayed in the extended reality space include: When the extended reality device is in a first working mode, a relative posture locking option is displayed in the extended reality space; the first working mode includes a completely black mode or a perspective mode.

7. The method according to claim 1, characterized in that include: In response to an instruction to switch the working mode of the extended reality device to the second working mode, setting the relative posture between the user's head and the broadcast control component to be in an unlocked state; When the relative position between the user's head and the broadcast control component is in an unlocked state, the broadcast control component does not move with the user's head, and the relative position between the user's head and the broadcast control component is variable; the second working mode includes a virtual viewing mode.

8. An interactive device, characterized in that: include: Display module, used to display the playback control components and relative posture locking options in the extended reality space; The playback control component includes a video playback window and playback control options, and the video playback window is used to play the video; A setting module, configured to set the relative posture of the user's head and the broadcast control component to be in a locked state in response to a selection operation of the relative posture locking option; When the relative posture of the user's head and the broadcast control component is in a locked state, the broadcast control component moves with the user's head, and the relative posture of the user's head and the broadcast control component always remains unchanged.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images