View field area determination method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a field-of-view area determination method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: obtaining display delay data of a virtual reality VR device, and obtaining current first head rotation data of a wearer wearing the VR device; determining a first view field angle of a VR image according to the display delay data and the first head rotation data; and determining a view field area of the VR image based on the first view field angle and a predicted viewpoint position of the wearing user. By means of the technical scheme, the image display effect of the VR equipment can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a method, apparatus, electronic device, storage medium, and program product for determining a field of view. Background Technology

[0002] Currently, in some virtual reality (VR) scenarios, VR devices can display images based on the image content located within the viewport area of ​​the VR image, without needing to transmit and render the entire VR image, thus reducing the resources required for transmitting and rendering image data. However, in related technologies, the display effect of VR devices is not ideal. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, storage medium, and program product for determining the field of view, so as to improve the display effect of VR devices.

[0004] In a first aspect, embodiments of this disclosure provide a method for determining a field of view, including:

[0005] The display latency data of the virtual reality (VR) device is obtained, and the first head rotation data of the user wearing the VR device is obtained.

[0006] Based on the display delay data and the first head rotation data, the first field of view angle of the VR image is determined;

[0007] The field of view of the VR image is determined based on the first field of view angle and the predicted viewpoint position of the wearer.

[0008] Secondly, embodiments of this disclosure also provide a device for determining the field of view, comprising:

[0009] The data acquisition module is used to acquire display latency data of the virtual reality (VR) device and acquire the current first head rotation data of the user wearing the VR device.

[0010] An angle determination module is used to determine the first field of view angle of the VR image based on the display delay data and the first head rotation data;

[0011] The region determination module is used to determine the field of view region of the VR image based on the first field of view angle and the predicted viewpoint position of the wearer.

[0012] Thirdly, embodiments of this disclosure also provide an electronic device, including:

[0013] One or more processors;

[0014] Memory, used to store one or more programs.

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the field of view region as described in the embodiments of this disclosure.

[0016] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining the field of view region as described in embodiments of this disclosure.

[0017] Fifthly, embodiments of this disclosure also provide a computer program product that, when executed by a computer, enables the computer to implement the method for determining the field of view region as described in embodiments of this disclosure.

[0018] The method, apparatus, electronic device, storage medium, and program product for determining the field of view area provided in this disclosure dynamically adjust the field of view angle of the field of view area based on the display latency data of the VR device and the head rotation data of the user. Compared with the technical solution of setting the field of view angle of the field of view area to a fixed value, it can improve the accuracy of the determined field of view area, improve the display quality of the VR device, and reduce the proportion of frames with black borders during the display of the VR device, thereby improving the display effect of the VR device. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A flowchart illustrating a method for determining a field of view region provided in an embodiment of this disclosure;

[0021] Figure 2 A flowchart illustrating another method for determining the field of view region provided in an embodiment of this disclosure;

[0022] Figure 3 A structural block diagram of a field-of-view region determination device provided in an embodiment of this disclosure;

[0023] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0026] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0028] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0029] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0030] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0031] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0032] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0033] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0034] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0035] Figure 1 This is a flowchart illustrating a method for determining a field of view region according to an embodiment of the present disclosure. The method can be executed by a field of view region determining device, which can be implemented in software and / or hardware and can be configured in an electronic device, typically a computer, mobile phone, or tablet computer. The field of view region determining method provided in this disclosure is applicable to scenes where the field of view region of a virtual reality image to be transmitted is determined.

[0036] Traditional VR streaming and transmission methods typically require transmitting the entire VR scene image, resulting in significant bandwidth consumption. Furthermore, due to the limited field of view (FoV) of the user, it's impossible to view the entire VR scene content simultaneously, leading to substantial bandwidth waste when transmitting the complete image. Therefore, some related technologies have proposed a viewport-based VR streaming and transmission method. This method predicts the user's future viewpoint position, determines the FoV area they are likely to view, and renders and transmits only the content within that area, significantly reducing bandwidth requirements. This is particularly important for cloud-based VR services and wireless VR devices. Moreover, the viewport-based streaming and transmission strategy can concentrate more computing and graphics resources on rendering the viewport area, effectively improving image clarity and smoothness in that area, further enhancing the immersion and interactivity of the VR experience. The viewport is the visible area used by the VR device to display the VR image; the maximum screen size of the VR scene image actually displayed to the user in the VR device is generally the same as the size of this viewport.

[0037] The performance of viewport-based VR transmission solutions largely depends on viewpoint prediction algorithms. However, human viewpoint movement is highly random, making accurate viewpoint prediction a challenging task. To mitigate potential experience issues caused by prediction errors, such as black bars due to sudden head movements, viewport-based transmission solutions typically transmit a fixed-size image area larger than the original Field of View (FoV) (e.g., 90° x 90°), such as 134° x 126°. This area is often referred to as the extended field of view (e.g., the super-FOV). Expanding the super-FOV can alleviate the black bar problem to some extent, but inevitably leads to a decrease in image quality within the FoV. Conversely, shrinking the super-FOV, while improving image quality within the FoV, increases the risk of black bars, thus affecting the overall visual experience.

[0038] In view of this, the present disclosure proposes a method for determining the field of view region, which can adaptively adjust the size of the super-FoV region according to the current network conditions and the user's head movement state, thereby suppressing the black border phenomenon to a certain extent while ensuring the image quality of the FoV region, and achieving a better balance between image quality and black border phenomenon.

[0039] like Figure 1 As shown, the method for determining the field of view region provided in this embodiment may include:

[0040] S101. Obtain the display latency data of the virtual reality (VR) device and obtain the current first head rotation data of the user wearing the VR device.

[0041] The display latency data of the VR device can be understood as data describing the display latency of the VR device when displaying VR image content, such as the display delay of a VR image. Optionally, the display latency data includes the average display latency within the most recent preset time period. For example, the display latency data of the VR device can be the average display latency of the VR device within the most recent preset time period, such as the average display latency of the VR device at specific times in the most recent n field-of-view areas. This representation of display latency can be set as needed. For example, this display latency can be the motion-to-photon latency (MTP latency) of the VR device, which will be used as an example below. MTP latency refers to the time delay from the movement of the user's head to the display of the corresponding image content on the screen. A VR image can be understood as a panoramic image to be displayed by the VR device, such as panoramic video and / or panoramic pictures, etc.

[0042] The user wearing the VR device can be understood as someone wearing the VR equipment. The first head rotation data can be understood as the actual head rotation data of the user at the current moment (e.g., the current field of view determination period). This actual head rotation data can be used to characterize the user's actual head rotation. Optionally, the first head rotation data includes head rotation speed, such as the instantaneous head rotation speed of the user at the current moment. This head rotation speed can include the yaw rate and / or pitch rate of the user's head. The yaw rate can be understood as the angular velocity of the user's head rotation around the vertical axis, such as the angular velocity of the user's head rotating left and right. The pitch rate can be understood as the angular velocity of the user's head rotation around the left and right axes, such as the angular velocity of the user's head rotating forward and backward.

[0043] In this embodiment, when the conditions for determining the field of view are met, such as when the determination time of the field of view is reached, the display latency data and first head motion data of the VR device can be obtained, such as calculating the average MTP latency of the VR device in the most recent n moments and the yaw rate and pitch rate of the user's head at the current moment, so as to redetermine the field of view of the VR image based on this.

[0044] Taking the display latency data as the average MTP latency and the first head rotation data including yaw rate and pitch rate as an example, for instance, assuming the current time is time t, the rotation angle of the user's head at the current time is (Yaw... t Pitch t The rotation angle of the user's head at the previous moment was (Yaw). t-1 Pitch t-1 The average MTP latency of VR devices is... sma ) can be:

[0045] MTP sma =(mtpt+mtpt-1+mtpt-2+...+mtpt-(n-1)) / n

[0046] Wherein, mtpt-(n-1) to mtpt represent the MTP latency of the VR device from time t-(n-1) to time t, respectively. The value of n can be preset, such as setting n to 10 or 15, etc.; it can also be adjusted according to the actual network conditions or needs. For example, the specific value of n can be adjusted manually, or the specific value of n can be automatically adjusted according to network stability and / or the display effect of the VR device's image content (such as the proportion of abnormal display conditions such as black borders). This embodiment does not limit this. It should be noted that although this embodiment uses the moving average method to calculate the average MTP latency of the VR device as an example, the calculation method of the average MTP latency of the VR device is not limited to this. For example, this embodiment can also use exponential average, weighted average, or other methods to calculate the average MTP latency of the VR device.

[0047] The yaw rate S of the user's head at the current moment. yaw and pitch angular velocity S pitch They are respectively:

[0048] S yaw = (Yawt - Yawt-1) / (Tt - T(t-1))

[0049] S pitch =(Pitch t -Pitch t-1 ) / (Tt-T(t-1))

[0050] Where Tt is the current time value, and T(t-1) is the previous time value (i.e., time t-1).

[0051] S102. Determine the first field of view angle of the VR image based on the display delay data and the first head rotation data.

[0052] In this context, the first field of view (FOV) of a VR image can be understood as the FOV determined based on the display latency data of the VR device and the first head rotation data of the user. The FOV is an angular parameter describing the visible range of an optical device (such as a lens or camera), determining the size of the spatial area the device can capture. In a VR scene, this FOV can be understood as the angle between the edge of the FOV area and the line connecting the observation point (such as the user's eye). The FOV can be understood as the angular value of the FOV. For example, the first FOV may include a first FOV in the horizontal direction (such as the yaw angle direction) and / or a first FOV in the vertical direction (such as the pitch angle direction).

[0053] Specifically, after obtaining the network latency data of the VR device and the first head rotation data of the user, the first field of view angle of the VR image can be determined based on the display latency data and the first head rotation data. For example, the first field of view angle of the VR image in the horizontal direction can be determined based on the display latency data and the user's head rotation data in the left-right direction (such as the yaw angle direction), and / or, the first field of view angle of the VR image in the vertical direction can be determined based on the display latency data and the user's head rotation data in the front-back direction (such as the pitch angle direction).

[0054] In this embodiment, the method for determining the first field of view angle of the VR image is not limited. For example, the first field of view angle of the VR image can be calculated according to a preset calculation method based on the network latency data of the VR device and the first head motion data of the user. Alternatively, the network latency data of the VR device and the first head motion data of the user can be input into a pre-trained field of view determination model. The field of view determination model can then determine the field of view angle based on the network latency data and the first head motion data, and the field of view angle output by the field of view determination model can be obtained as the first field of view angle of the VR image.

[0055] S103. Determine the field of view area of ​​the VR image based on the first field of view angle and the predicted viewpoint position of the user wearing the device.

[0056] The predicted viewpoint position of the wearer can be understood as the viewpoint position of the wearer at the next moment, determined through prediction. Viewpoint position can be understood as the location of the wearer's focus point, such as the angular coordinates (e.g., yaw and pitch angles) of the target point viewed by the user in the VR image. The field of view can be understood as the area in the VR image to be transmitted to the VR device and / or rendered by the VR device. For example, this field of view can be an extended field of view, whose size can be larger than the original field of view actually displayed by the VR device.

[0057] For example, the field of view of a VR image can be determined based on a first field of view angle and the predicted viewpoint position of the user wearing the device. For instance, the predicted viewpoint position of the user wearing the device can be used as the center of the field of view area, and the predicted viewpoint position can be rotated to the left and right by 1 / 2 of the first field of view angle in the horizontal direction, and / or the predicted viewpoint position can be rotated upward and downward by 1 / 2 of the first field of view angle in the vertical direction, thereby obtaining the field of view area of ​​the VR image.

[0058] In this embodiment, the predicted viewpoint position of the wearer can be obtained by predicting the viewpoint of the wearer, and the specific prediction method is not limited.

[0059] In some examples, the wearer's viewpoint position can be predicted based on the wearer's current actual viewpoint position and / or region center offset value to obtain the wearer's predicted viewpoint position. Optionally, before determining the field of view region of the VR image based on the first field of view angle and the wearer's predicted viewpoint position, the method further includes: predicting the wearer's viewpoint position based on the wearer's current actual viewpoint position and / or region center offset value to obtain the wearer's predicted viewpoint position.

[0060] The user's current true viewpoint position can be understood as the user's actual viewpoint position at the current moment. This position can be determined and returned by the VR device. For example, the center point of the original field of view (i.e., the field of view before expansion) displayed by the VR device at the current moment can be obtained as the user's current true viewpoint position. The region center offset value can be understood as the value that needs to be further offset when predicting the viewpoint position. This region center offset value can correspond to the user's predicted head rotation speed. The user's predicted head rotation speed can be understood as the predicted head rotation speed of the user in the future (such as the next moment), and the specific prediction method is not limited.

[0061] For example, when predicting the viewpoint position of a user, we can disregard the user's potential future head movements and predict the user's viewpoint position based on the user's current actual viewpoint position. This predicted viewpoint position may or may not be the same as the user's current actual viewpoint position.

[0062] For example, when testing the viewpoint position of a user, potential future head movements can be considered. Based on the user's current true viewpoint position and the region center offset value, the user's viewpoint position can be predicted to further improve the accuracy of the predicted viewpoint position. For instance, the user's current true viewpoint position can be offset based on the region center offset value, and the predicted viewpoint position can be predicted based on the offset position. Here, the predicted field of view position and the viewpoint position obtained from the offset processing can be the same or different.

[0063] For example, assuming the user's actual viewpoint position at the current moment (such as the actual super FOV center position of the current VR device) is (Oldy, Oldp) and the region center offset value is (ORY, ORP), then the viewpoint position (Newy, Newp) after offset processing can specifically be (Oldy+ORY, Oldp+ORP).

[0064] In this embodiment, after determining the field of view of the VR image, VR streaming and transmission can be performed based on this field of view until the field of view of the VR image changes, such as until the field of view of the VR image is determined at the next moment. For example, image content of one or more VR images to be displayed (such as video frames of panoramic videos or panoramic images) located within this field of view can be acquired, VR streaming can be performed based on this image content to generate a VR data stream, and this VR data stream can be sent to the VR device, thereby realizing VR streaming and transmission. After receiving this VR data stream, the VR device can perform image rendering and display of image content based on this VR data stream. In this case, optionally, after determining the field of view of the VR image, the method further includes: generating a VR data stream based on the image content of the VR image located within the field of view, and sending the VR data stream to the VR device, wherein the VR data stream is used by the VR device to display image content.

[0065] The method for determining the field of view area provided in this embodiment acquires the display latency data of the VR device and the first head rotation data of the user wearing the VR device; based on the display latency data and the first head rotation data, a first field of view angle of the VR image is determined; based on the first field of view angle and the predicted viewpoint position of the user, the field of view area of ​​the VR image is determined. This embodiment utilizes the above technical solution to dynamically adjust the field of view angle of the field of view area based on the display latency data of the VR device and the head rotation data of the user. Compared with technical solutions that set the field of view angle of the field of view area to a fixed value, this can improve the accuracy of the determined field of view area, improve the image display quality of the VR device, and reduce the proportion of frames with black borders during VR device image display, thereby improving the image display effect of the VR device.

[0066] Figure 2 This is a flowchart illustrating another method for determining the field of view region provided in this embodiment. The solution in this embodiment can be combined with one or more optional solutions in the above embodiments. Optionally, determining the first field of view angle of the VR image based on the display delay data and the first head rotation data includes: determining a first field of view angle adjustment value for the VR image based on the display delay data and the first head rotation data; adjusting a second field of view angle based on the first field of view angle adjustment value to obtain the first field of view angle of the VR image, wherein the second field of view angle is a preset field of view angle.

[0067] Correspondingly, such as Figure 2 As shown, the method for determining the field of view region provided in this embodiment may include:

[0068] S201. Obtain the display latency data of the virtual reality (VR) device and obtain the current first head rotation data of the user wearing the VR device.

[0069] S202. Based on the display delay data and the first head rotation data, determine the first field of view angle adjustment value of the VR image.

[0070] The first field-of-view angle adjustment value can be understood as a field-of-view angle adjustment value determined based on the display latency data of the VR device and the first head rotation data, such as a field-of-view angle increase value. For example, the first field-of-view angle adjustment value may include a horizontal first field-of-view angle adjustment value and / or a vertical first field-of-view angle adjustment value. The first field-of-view angle adjustment value can be used to adjust a pre-set second field-of-view angle.

[0071] Specifically, after obtaining the display latency data of the VR device and the first head rotation data of the user, the first field of view angle adjustment value can be determined based on this display latency data and the first head rotation data. For example, the first field of view angle adjustment value can be determined based on this display latency data and the first head rotation data through a pre-set adjustment value determination method and / or a pre-trained field of view determination model.

[0072] In some implementations, the display latency data includes the average display latency within the most recent preset time period, the first head rotation data includes the head rotation speed, and the step of determining the first field of view adjustment value of the VR image based on the display latency data and the first head rotation data includes: calculating the product of the display latency data, the first head rotation data and a preset coefficient as the first field of view adjustment value of the VR image.

[0073] For example, the display latency data of the VR device, the first head rotation data of the user in the Yaw direction, and the product of a preset coefficient can be calculated as the first field of view adjustment value in the horizontal direction (such as the Yaw direction); and / or, the display latency data of the VR device, the first head rotation data of the user in the Pitch direction, and the product of a preset coefficient can be calculated as the first field of view adjustment value in the vertical direction (such as the Pitch direction).

[0074] The average MTP latency of VR devices is shown in the display latency data. sma The first head rotation data includes the wearer's current yaw rate S. yaw and pitch angular velocity S pitchTaking Ratio as a preset coefficient as an example, the movement range (Move Range Yaw, MRY) in the Yaw direction can be calculated as the first field of view angle adjustment value in the Yaw direction, and / or the movement range (Move Range Pitch, MRP) in the Pitch direction can be calculated as the first field of view angle adjustment value in the Pitch direction, thereby determining the first field of view angle adjustment value.

[0075] MRY=S yaw ×MTP sma ×Ratio

[0076] MRP = S pitch ×MTP sma ×Ratio

[0077] The preset coefficient Ratio is a pre-set range scaling factor, which can be adjusted according to network status or other information. For example, when the display latency data increases, the value of Ratio can be increased appropriately to reduce the probability of black borders appearing when the VR device displays the image; when the display latency data decreases, the value of Ratio can be decreased appropriately to reduce the determined first field of view angle, thereby reducing the size of the field of view area.

[0078] In some examples, multiple data ranges for display latency can be preset, and different data ranges can correspond to different preset coefficients. For example, when the display latency data in a certain data range is greater than the display latency data in another data range, the preset coefficient corresponding to this data range can be greater than the preset coefficient corresponding to the other data range. Taking three data ranges [0, d1) and [d2, +∞) as an example, the preset coefficient corresponding to the data range [0, d1) can be Ratio1, the preset coefficient corresponding to the data range [d1, d2) can be Ratio2, and the preset coefficient corresponding to the data range [d2, +∞) can be Ratio3, where Ratio1 < Ratio2 < Ratio3. Among them, d1, d2, Ratio1, Ratio2 and Ratio3 can all be set as needed. For example, d1 can be set to 75ms, 80ms or 85ms, d2 can be set to 87ms, 90ms or 95ms, Ratio1 can be set to 25 or 30, Ratio2 can be set to 35 or 40, and Ratio3 can be set to 45 or 50.

[0079] Therefore, a preset coefficient for calculating the first field-of-view adjustment value can be determined based on the data interval where the display delay data is located. When the data intervals where the display delay data is located are different, the value of this preset coefficient can be different. In this case, optionally, the preset coefficient corresponds to the data interval where the display delay data is located.

[0080] S203. Adjust the second field of view according to the first field of view angle adjustment value to obtain the first field of view angle of the VR image, wherein the second field of view angle is a preset field of view angle.

[0081] The second field of view angle is a preset field of view angle, such as a pre-set default field of view angle. Taking the field of view area determined in this embodiment as an extended field of view area as an example, the second field of view angle can be the default field of view angle of the original field of view area before the area expansion. The area size of this original field of view area can be greater than or equal to the area size of the field of view area actually displayed by the VR device.

[0082] Specifically, after obtaining the first field of view angle adjustment value, the preset second field of view angle can be adjusted according to this first field of view angle adjustment value, and the adjusted second field of view angle can be used as the first field of view angle.

[0083] In this embodiment, the method of adjusting the second field of view angle according to the first field of view angle adjustment value is not limited.

[0084] In some implementations, the second field of view can be directly adjusted based on the first field of view adjustment value, such as by increasing the second field of view by a set multiple of the first field of view adjustment value, thus obtaining the first field of view. Optionally, in this case, adjusting the second field of view based on the first field of view adjustment value includes: adjusting the second field of view based on the first field of view adjustment value. The set multiple can be set as needed; for example, the set multiple can be a value such as 1, 1.5, 2, or 3.

[0085] In some examples, taking a multiplier of 2 as an example, the default FoV region corresponding to the second field of view angle can be symmetrically increased by a certain value (e.g., MRY) in the yaw direction, and / or the default FoV region corresponding to the second field of view angle can be symmetrically increased by a certain value (e.g., MRP) in the pitch direction. In this case, the first field of view angle RY (i.e., Range Yaw, yaw angle range) in the yaw direction and the first field of view angle RP (i.e., Range Pitch, pitch angle range) in the pitch direction can be calculated by the following formula:

[0086] RY = FoV yaw +MRY×2

[0087] RP = FoV pitch +MRP×2

[0088] Among them, FoV yaw This is the second field of view angle in the yaw direction, i.e., the default FoV size in the yaw direction. Its specific value can be set as needed, for example, it can be set to 100°, 105°, or 110°, etc. pitch This is the second field of view angle in the pitch direction, i.e., the default FoV size in the pitch direction. Its specific value can be set as needed, for example, it can be set to 100°, 105°, or 110°, etc. yaw With FoV pitch They can be the same or different.

[0089] In the above implementation, the super-FOV range can be dynamically adjusted based on the user's current head movement speed (i.e., the first head rotation data) and network status to achieve a better balance between FoV image quality and black border rate. This network status may include, but is not limited to, latency data, such as display latency data, which can be adjusted appropriately as needed. Taking MTP latency as an example, when MTP increases or the user's head movement speed is fast, the super-FOV range (i.e., the first field of view angle) can be appropriately increased to reduce black borders; when MTP decreases or the user's head movement speed is slow, the super-FOV range can be appropriately decreased to improve image quality.

[0090] In some implementations, after obtaining the first field of view angle adjustment value, the first field of view angle adjustment value can be further adjusted based on the wearer's possible future rotation to obtain a second field of view angle adjustment value. The second field of view angle is then adjusted based on the second field of view angle adjustment value to fully consider the physiological characteristics and behavioral patterns of human head movements, further reduce the size of the determined field of view area, and further improve the FoV image quality.

[0091] Optionally, adjusting the second field of view angle based on the first field of view angle adjustment value includes: adjusting the first field of view angle adjustment value based on the wearer's predicted head rotation direction and region center offset value to obtain a second field of view angle adjustment value; and adjusting the second field of view angle based on the second field of view angle adjustment value, wherein the region center offset value corresponds to the wearer's predicted head rotation speed.

[0092] The predicted head rotation direction of the wearer can be the predicted possible head rotation direction of the wearer in the future (such as at the next moment). The second field of view adjustment value can be understood as the first field of view adjustment value adjusted based on the predicted head rotation direction of the wearer and the region center offset value.

[0093] Specifically, the first field-of-view angle adjustment value can be adjusted based on the user's predicted head rotation direction and the region center offset value to obtain the second field-of-view angle adjustment value. For example, based on the user's predicted head rotation direction, the first field-of-view angle adjustment value in this predicted head rotation direction can be increased, and / or, based on the region center offset value, the first field-of-view angle adjustment value in the opposite direction of this predicted head rotation direction can be decreased to obtain the second field-of-view angle adjustment value.

[0094] Taking the reduction of the first field-of-view angle adjustment value in the opposite direction of the predicted head rotation direction based on the region center offset value as an example, in some examples, assuming that the predicted head rotation direction of the wearer is to the upper left, it can be determined that the wearer rotates to the left in the Yaw direction and upward in the Pitch direction. In this case, the first field-of-view angle adjustment value to the left in the Yaw direction can be kept unchanged, that is, the second field-of-view angle adjustment value to the left in the Yaw direction RY1 = MRY, and the first field-of-view angle adjustment value to the right in the Yaw direction can be reduced by this region center offset value ORY, that is, the second field-of-view angle adjustment value to the right in the Yaw direction RY2 = MRY – ORY; and the first field-of-view angle adjustment value to the upward in the Pitch direction can be kept unchanged, that is, the second field-of-view angle adjustment value to the upward in the Pitch direction RP1 = MRP, and the first field-of-view angle adjustment value to the downward in the Pitch direction can be reduced by this region center offset value ORY, that is, the second field-of-view angle adjustment value to the downward in the Pitch direction RP2 = MRP – ORP.

[0095] After obtaining the second field of view adjustment value, the second field of view angle can be adjusted based on this value to obtain the first field of view angle. For example, the sum of the second field of view angle in the yaw direction and the second field of view angle adjustment values ​​in the yaw direction (including left and right) can be calculated as the first field of view angle in the yaw direction; and / or, the sum of the second field of view angle in the pitch direction and the second field of view angle adjustment values ​​in the pitch direction (including up and down) can be calculated as the first field of view angle in the pitch direction.

[0096] In the above embodiments, head motion prediction can be performed on the wearer to obtain head rotation data such as the predicted head rotation speed and predicted head rotation direction; and / or, a region center offset value can be calculated based on the predicted head rotation speed and a first field of view angle adjustment value. Optionally, in this case, the method for determining the field of view region further includes: performing head motion prediction on the wearer to obtain second head rotation data of the wearer, the second head rotation data including the predicted head rotation speed and predicted head rotation direction; and calculating the region center offset value based on the predicted head rotation speed and the first field of view angle adjustment value.

[0097] The second head rotation data can be understood as predicted head rotation data that the user is about to experience (at the next moment). This data may include predicted head rotation speed and predicted head rotation direction, and may further include predicted head rotation range. The predicted head rotation speed can be understood as the predicted head rotation speed that the user is about to experience. The predicted head rotation direction can be understood as the predicted head rotation direction that the user is about to experience, such as the user's rotation direction in the yaw and / or pitch directions. The predicted head rotation range can be understood as the predicted and determined maximum rotation range that the user is physiologically achievable in the current state, for example, the maximum physiologically achievable rotation range in the yaw and / or pitch directions in the current state, such as the maximum physiologically achievable rotation angle.

[0098] For example, a head motion prediction model can be used to predict the head motion that the wearer will be performing (as in the next moment), such as predicting the wearer's head rotation speed, head rotation direction, and / or head rotation range, etc., as second head rotation data.

[0099] After predicting the user's head rotation speed, the region center offset can be calculated based on this predicted head rotation speed and the first field of view angle adjustment value. The specific calculation method for the region center offset is not limited, as long as it is positively correlated with the predicted head rotation speed and less than MRY. In some examples, it can be ensured that the region center offset is positively correlated with the predicted head rotation speed and less than MRY / 2 to avoid black border areas appearing in the opposite direction. The following explanation uses this case as an example.

[0100] Taking the calculation of the region center offset value based on the hyperbolic tangent function as an example, assuming the predicted head movement speed of the wearer in the Yaw direction is Speed yaw The predicted head movement speed of the wearer in the Pitch direction is Speed. pitch If the area center offset in the Yaw direction is ORY (Offset Range Yaw, the range of yaw angle compensation) and the area center offset in the Pitch direction is ORP (Offset Range Pitch, the range of pitch angle compensation), then:

[0101] ORY = tanh(Speed yaw )×(MRY / 2)

[0102] ORP = tanh(Speed) pitch )×(MRP / 2)

[0103] In the above embodiments, when adjusting the first field of view angle based on the wearer's predicted head rotation direction and the region center offset value, the wearer's predicted head rotation range may or may not be considered.

[0104] In some implementations, the predicted head rotation range of the wearer can be considered to further improve the practicality of the determined field of view. Optionally, in this case, the second head rotation data also includes the predicted head rotation range, and adjusting the first field of view angle adjustment value based on the wearer's predicted head rotation direction and the region center offset value includes: adjusting the first field of view angle adjustment value within the wearer's predicted head rotation range based on the wearer's predicted head rotation direction and the region center offset value.

[0105] For example, the first field of view adjustment value can be adjusted firstly based on the predicted head rotation direction and the region center offset value of the wearer. If the adjusted first field of view adjustment value is within the predicted head rotation range in the corresponding direction, it can be determined as the second field of view adjustment value in this direction. If the adjusted first field of view adjustment value is not within the predicted head rotation range in the corresponding direction, it can be further adjusted to be within the predicted head rotation range, such as adjusting it to the nearest endpoint value within the predicted head rotation range, to obtain the second field of view adjustment value in this direction.

[0106] Assuming the predicted head rotation range corresponding to the second field-of-view angle adjustment value RY1 to the left in the Yaw direction is 0≤RY1≤MAX yaw1 After obtaining the adjusted first field-of-view angle value to the left in the Yaw direction, if this adjusted first field-of-view angle value is within this range, it can be used as the second field-of-view angle adjustment value to the left in the Yaw direction; if this adjusted first field-of-view angle adjustment value is not within this range, for example, if this adjusted first field-of-view angle adjustment value is greater than MAX... yaw1 Then it can be further adjusted to MAX. yaw1 This serves as the second field-of-view angle adjustment value to the left in the Yaw direction. The adjustment method for the first field-of-view angle adjustment value in other directions is similar to that for the first field-of-view angle adjustment value to the left in the Yaw direction. For details, please refer to the adjustment process of this first field-of-view angle adjustment value. This embodiment will not repeat the details here.

[0107] In the above implementation, for example, the center position of the super FoV can be dynamically adjusted first based on the head motion model to achieve an asymmetrical distribution of the extra area in the left-right and up-down directions of the FoV: the extra area is expanded in the predicted motion direction to cover a larger potential FoV offset; the extra area is reduced in the opposite direction to avoid transmitting redundant areas. Specifically, the head motion model can be used to predict the direction and speed of the wearer's next head movement, and the possible FoV offset can be estimated accordingly. At the same time, the center position of the super FoV is shifted accordingly to dynamically adjust the distribution of the extra area around the FoV. For example, when the wearer's head moves to the left, the center of the super FoV can be shifted to the left, expanding the extra area on the left, and / or reducing the extra area on the right, thereby more accurately matching the change in viewing angle. In extreme cases, the extra area in the opposite direction can even be reduced to 0, minimizing the super FoV range to improve FoV image quality.

[0108] Furthermore, the physiological limitations and characteristics of human head movement can be further considered, and head motion models can be used to further improve the rationality of calculating the super-FoV region range. Studies have shown that the range of angles that the head can autonomously rotate under normal circumstances is significantly limited, and this limitation will change accordingly due to factors such as the wearer's body posture, inertial response, and / or the wearing device. Therefore, a comprehensive analysis of the wearer's current head movement state (including velocity, acceleration, direction, etc.) can be performed using a head motion model to dynamically estimate the maximum physiologically achievable head rotation angle (i.e., the predicted head rotation range) under this state. This angle is used as a constraint to limit the maximum expansion value of the additional region in each direction within the super-FoV range, thereby improving the rationality of the region allocation. For example, when the wearer's maximum leftward head rotation angle is 30° in a certain posture, the additional region on the left side of the FoV can be expanded by a maximum of 30°, avoiding the transmission of redundant regions.

[0109] Therefore, by combining the wearer's actual movement state and physiological accessibility, the center position and boundaries of the super-FoV are dynamically adjusted to achieve a more refined super-FoV range allocation. The above implementation method, while ensuring viewport quality, can further reduce the size of the super-FoV range, improve image quality and transmission efficiency, and provide strong technical support for VR streaming and transmission.

[0110] In the above implementation, a head motion prediction model can be used to predict the direction and speed of the user's upcoming head movements, as well as the maximum physiologically achievable rotation angle in the current state. It should be noted that the head motion prediction model can be flexibly configured as needed. For example, on mobile platforms with limited computing power, a linear prediction model with lower computational overhead can be used; while on high-performance platforms with sufficient computing power, a deep neural network model can be introduced to improve the prediction accuracy of head movement trends and physiological limits, thereby achieving more accurate calculations beyond the FoV range.

[0111] In this embodiment, when adjusting the second field of view based on the first field of view adjustment value, an angle range for the second field of view may or may not be set. In some implementations, an angle range for the second field of view can be set, so that the second field of view can be adjusted within this angle range based on the first field of view adjustment value, ensuring that the adjusted first field of view is within this angle range and avoiding the first field of view being too large or too small. In this case, optionally, adjusting the second field of view based on the first field of view adjustment value includes: adjusting the second field of view within a preset angle range based on the first field of view adjustment value.

[0112] The preset angle range can be understood as the angle adjustment range preset for the second field of view, or it can be understood as the angle range preset for the first field of view. For example, the preset angle range may include the angle adjustment range in the yaw direction and the angle adjustment range in the pitch direction. These two angle adjustment ranges may be the same or different, and the endpoint values ​​of these two angle adjustment ranges can be set as needed. This embodiment does not limit this.

[0113] For example, the second field of view can be adjusted first based on the first field of view adjustment value to obtain the adjusted second field of view. If the adjusted second field of view is within a preset angle range, it can be determined as the first field of view. If the adjusted second field of view is not within the preset angle range, it can be further adjusted to the corresponding preset angle range, such as adjusting it to the endpoint value closest to the corresponding preset angle range to obtain the first field of view.

[0114] Taking the adjustment of the second field of view angle in the Yaw direction as an example, assuming that the preset angle range corresponding to the first field of view angle RY in the Yaw direction is A1≤RY≤A2, the second field of view angle in the Yaw direction can be adjusted first according to the adjustment value of the first field of view angle in the Yaw direction to obtain the adjusted second field of view angle in the Yaw direction. If the adjusted second field of view angle in the Yaw direction is within the above-mentioned preset angle range, the adjusted second field of view angle in the Yaw direction can be used as the first field of view angle in the Yaw direction. If the adjusted second field of view angle in the Yaw direction is less than A1, the adjusted second field of view angle in the Yaw direction can be further adjusted to A1 and used as the first field of view angle in the Yaw direction. If the adjusted second field of view angle in the Yaw direction is greater than A2, the adjusted second field of view angle in the Yaw direction can be further adjusted to A2 and used as the first field of view angle in the Yaw direction.

[0115] The method for adjusting the second field of view angle in the Pitch direction is similar to the method for adjusting the second field of view angle in the Yaw direction. For details, please refer to the method for adjusting the second field of view angle in the Yaw direction. This embodiment will not repeat the details here.

[0116] S204. Determine the field of view area of ​​the VR image based on the first field of view angle and the predicted viewpoint position of the user wearing the device.

[0117] The method for determining the field of view area provided in this embodiment determines the first field of view angle adjustment value of the VR image based on display delay data and first head rotation data; and adjusts the preset second field of view angle according to this first field of view angle adjustment value to obtain the first field of view angle of the VR image. This can further improve the accuracy of the determined first field of view angle, thereby further improving the accuracy of the determined field of view area and improving the image display effect of the VR device.

[0118] Figure 3 This is a structural block diagram of a field-of-view region determination device provided in an embodiment of this disclosure. The device can be implemented by software and / or hardware, and can be configured in an electronic device, typically a computer, mobile phone, or tablet computer. It can determine the field-of-view region of a virtual reality image to be transmitted by executing a field-of-view region determination method. Figure 3 As shown, the field-of-view determination device provided in this embodiment may include: a data acquisition module 301, an angle determination module 302, and a region determination module 303, wherein,

[0119] The data acquisition module 301 is used to acquire display latency data of the virtual reality (VR) device and acquire the first head rotation data of the user wearing the VR device.

[0120] Angle determination module 302 is used to determine the first field of view angle of the VR image based on the display delay data and the first head rotation data;

[0121] The region determination module 303 is used to determine the field of view region of the VR image based on the first field of view angle and the predicted viewpoint position of the wearer.

[0122] The field-of-view determination device provided in this embodiment acquires display latency data of the VR device and the current first head rotation data of the user wearing the VR device through a data acquisition module; the angle determination module determines the first field-of-view angle of the VR image based on the display latency data and the first head movement data; and the region determination module determines the field-of-view area of ​​the VR image based on the first field-of-view angle and the predicted viewpoint position of the user. This embodiment utilizes the above technical solution to dynamically adjust the field-of-view angle of the field-of-view area based on the display latency data of the VR device and the head rotation data of the user. Compared with technical solutions that set the field-of-view angle of the field-of-view area to a fixed value, this improves the accuracy of the determined field-of-view area, enhances the image display quality of the VR device, and reduces the proportion of frames with black borders during VR image display, thereby improving the image display effect of the VR device.

[0123] Optionally, the angle determination module 302 includes: an adjustment value determination unit, configured to determine a first field-of-view angle adjustment value for the VR image based on the display delay data and the first head rotation data; and an angle adjustment unit, configured to adjust a second field-of-view angle based on the first field-of-view angle adjustment value to obtain the first field-of-view angle of the VR image, wherein the second field-of-view angle is a preset field-of-view angle.

[0124] Optionally, the display delay data includes the average display delay within the most recent preset time period, and the adjustment value determination unit can be specifically used to: calculate the product of the display delay data, the first head rotation data and the preset coefficient as the first field of view adjustment value of the VR image.

[0125] Optionally, the preset coefficient corresponds to the data range in which the display delay data is located.

[0126] Optionally, the angle adjustment unit can be used to: adjust the second field of view angle according to the first field of view angle adjustment value; or, adjust the first field of view angle adjustment value according to the predicted head rotation direction and the region center offset value of the wearer to obtain the second field of view angle adjustment value; and adjust the second field of view angle according to the second field of view angle adjustment value, wherein the region center offset value corresponds to the predicted head rotation speed of the wearer.

[0127] Furthermore, the device for determining the field of view area may further include: a rotation prediction module, used to predict the head movement of the wearer to obtain second head rotation data of the wearer, the second head rotation data including predicted head rotation speed and predicted head rotation direction; and an offset value calculation module, used to calculate the center offset value of the area based on the predicted head rotation speed and the first field of view angle adjustment value.

[0128] Optionally, the second head rotation data also includes a predicted head rotation range, and the angle adjustment unit can be specifically used to: adjust the first field of view angle adjustment value within the predicted head rotation range of the wearer based on the predicted head rotation direction and the region center offset value of the wearer.

[0129] Optionally, the angle determination module 302 can be used to: adjust the second field of view angle within a preset angle range according to the first field of view angle adjustment value.

[0130] Furthermore, the device for determining the field of view region may further include: a viewpoint prediction module, used to predict the viewpoint position of the wearer based on the wearer's current real viewpoint position and / or region center offset value before determining the field of view region of the VR image based on the first field of view angle and the wearer's predicted viewpoint position, thereby obtaining the wearer's predicted viewpoint position.

[0131] Furthermore, the device for determining the field of view may further include: a data stream sending module, configured to generate a VR data stream based on the image content of the VR image located within the field of view after determining the field of view of the VR image, and send the VR data stream to the VR device, wherein the VR data stream is used by the VR device to display image content.

[0132] The field-of-view determination apparatus provided in this disclosure can execute the field-of-view determination method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the field-of-view determination method. Technical details not described in detail in this embodiment can be found in the field-of-view determination method provided in any embodiment of this disclosure.

[0133] The following is for reference. Figure 4This illustration shows a structural diagram of an electronic device (e.g., a server) 400 suitable for implementing embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0134] like Figure 4 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. Processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0135] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0136] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a display via communication device 409, or installed from storage device 408, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0137] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0138] In some implementations, the client and server may communicate using any currently known or future-developed display protocol such as HTTP (Hypertext Transfer Protocol) and may interconnect with digital data communication (e.g., a communication display) of any form or medium. Examples of communication displays include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end displays (e.g., ad hoc end-to-end displays), as well as any currently known or future-developed displays.

[0139] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0140] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire display latency data of a virtual reality (VR) device and acquire first head rotation data of a user wearing the VR device; determine a first field of view angle of a VR image based on the display latency data and the first head rotation data; and determine the field of view region of the VR image based on the first field of view angle and the predicted viewpoint position of the user.

[0141] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any kind of display—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0143] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of modules do not, in some cases, constitute a limitation on the unit itself.

[0144] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0145] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0146] According to one or more embodiments of this disclosure, Example 1 provides a method for determining the field of view region, including:

[0147] The display latency data of the virtual reality (VR) device is obtained, and the first head rotation data of the user wearing the VR device is obtained.

[0148] Based on the display delay data and the first head rotation data, the first field of view angle of the VR image is determined;

[0149] The field of view of the VR image is determined based on the first field of view angle and the predicted viewpoint position of the wearer.

[0150] According to one or more embodiments of this disclosure, Example 2 describes the method described in Example 1, wherein determining the first field of view angle of the VR image based on the display delay data and the first head rotation data includes:

[0151] Based on the display delay data and the first head rotation data, determine the first field of view adjustment value of the VR image;

[0152] The second field of view is adjusted according to the first field of view adjustment value to obtain the first field of view angle of the VR image, wherein the second field of view angle is a preset field of view angle.

[0153] According to one or more embodiments of this disclosure, Example 3 describes the method according to Example 2, wherein the display latency data includes the average display latency within the most recent preset time period, the first head rotation data includes head rotation speed, and the step of determining a first field-of-view adjustment value for the VR image based on the display latency data and the first head rotation data includes:

[0154] The product of the display delay data, the first head rotation data, and the preset coefficient is calculated as the first field of view adjustment value of the VR image.

[0155] According to one or more embodiments of this disclosure, Example 4 describes the method according to Example 3, wherein the preset coefficient corresponds to the data range in which the display delay data is located.

[0156] According to one or more embodiments of this disclosure, Example 5 describes the method described in Example 2, wherein adjusting the second field of view angle based on the first field of view angle adjustment value includes:

[0157] Adjust the second field of view angle based on the first field of view angle adjustment value; or

[0158] Based on the wearer's predicted head rotation direction and region center offset value, the first field of view adjustment value is adjusted to obtain a second field of view adjustment value; based on the second field of view adjustment value, the second field of view angle is adjusted, wherein the region center offset value corresponds to the wearer's predicted head rotation speed.

[0159] According to one or more embodiments of this disclosure, Example 6 describes the method according to Example 5, the method further comprising:

[0160] The head motion of the wearer is predicted to obtain the second head rotation data of the wearer, which includes the predicted head rotation speed and the predicted head rotation direction.

[0161] The region center offset value is calculated based on the predicted head rotation speed and the first field of view angle adjustment value.

[0162] According to one or more embodiments of this disclosure, Example 7, based on the method of Example 6, further includes a predicted head rotation range in the second head rotation data. The adjustment of the first field-of-view angle adjustment value based on the predicted head rotation direction and region center offset value of the wearer includes:

[0163] Based on the user's predicted head rotation direction and the region center offset value, the first field of view angle adjustment value is adjusted within the user's predicted head rotation range.

[0164] According to one or more embodiments of this disclosure, Example 8 describes the method according to any of Example 2, wherein adjusting the second field of view based on the first field of view adjustment value includes:

[0165] Based on the first field of view angle adjustment value, the second field of view angle is adjusted within a preset angle range.

[0166] According to one or more embodiments of this disclosure, Example 9, based on any one of Examples 1-8, further includes, before determining the field of view region of the VR image based on the first field of view angle and the predicted viewpoint position of the wearer:

[0167] Based on the wearer's current true viewpoint position and / or region center offset value, the wearer's viewpoint position is predicted to obtain the wearer's predicted viewpoint position.

[0168] According to one or more embodiments of this disclosure, Example 10, following the determination of the field of view region of the VR image, further includes the following method according to any one of Examples 1-8:

[0169] A VR data stream is generated based on the image content of the VR image located within the field of view, and the VR data stream is sent to the VR device for the VR device to display the image content.

[0170] According to one or more embodiments of this disclosure, Example 11 provides a field-of-view region determination apparatus, comprising:

[0171] The data acquisition module is used to acquire display latency data of the virtual reality (VR) device and acquire the current first head rotation data of the user wearing the VR device.

[0172] An angle determination module is used to determine the first field of view angle of the VR image based on the display delay data and the first head rotation data;

[0173] The region determination module is used to determine the field of view region of the VR image based on the first field of view angle and the predicted viewpoint position of the wearer.

[0174] According to one or more embodiments of this disclosure, Example 12 provides an electronic device, including:

[0175] One or more processors;

[0176] Memory, used to store one or more programs.

[0177] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the field of view region as described in any of Examples 1-10.

[0178] According to one or more embodiments of the present disclosure, Example 13 provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a method for determining the field of view region as described in any of Examples 1-10.

[0179] According to one or more embodiments of this disclosure, Example 14 provides a computer program product that, when executed by a computer, causes the computer to implement a method for determining the field of view region as described in any of Examples 1-10.

[0180] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0181] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0182] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for determining a field of view, characterized in that, include: The display latency data of the virtual reality (VR) device is obtained, and the first head rotation data of the user wearing the VR device is obtained. Based on the display delay data and the first head rotation data, the first field of view angle of the VR image is determined; The field of view of the VR image is determined based on the first field of view angle and the predicted viewpoint position of the wearer.

2. The method according to claim 1, characterized in that, Determining the first field of view angle of the VR image based on the display delay data and the first head rotation data includes: Based on the display delay data and the first head rotation data, determine the first field of view adjustment value of the VR image; The second field of view is adjusted according to the first field of view adjustment value to obtain the first field of view angle of the VR image, wherein the second field of view angle is a preset field of view angle.

3. The method according to claim 2, characterized in that, The display latency data includes the average display latency within the most recent preset time period, and the first head rotation data includes the head rotation speed. Determining the first field-of-view adjustment value of the VR image based on the display latency data and the first head rotation data includes: The product of the display delay data, the first head rotation data, and the preset coefficient is calculated as the first field of view adjustment value of the VR image.

4. The method according to claim 3, characterized in that, The preset coefficient corresponds to the data range in which the display delay data is located.

5. The method according to claim 2, characterized in that, The step of adjusting the second field of view angle according to the first field of view angle adjustment value includes: Adjust the second field of view angle based on the first field of view angle adjustment value; or Based on the wearer's predicted head rotation direction and region center offset value, the first field of view adjustment value is adjusted to obtain a second field of view adjustment value; based on the second field of view adjustment value, the second field of view angle is adjusted, wherein the region center offset value corresponds to the wearer's predicted head rotation speed.

6. The method according to claim 5, characterized in that, The method further includes: The head motion of the wearer is predicted to obtain the second head rotation data of the wearer, which includes the predicted head rotation speed and the predicted head rotation direction. The region center offset value is calculated based on the predicted head rotation speed and the first field of view angle adjustment value.

7. The method according to claim 6, characterized in that, The second head rotation data also includes a predicted head rotation range. The adjustment of the first field of view angle adjustment value based on the predicted head rotation direction and region center offset value of the wearer includes: Based on the user's predicted head rotation direction and the region center offset value, the first field of view angle adjustment value is adjusted within the user's predicted head rotation range.

8. The method according to claim 2, characterized in that, The step of adjusting the second field of view angle according to the first field of view angle adjustment value includes: Based on the first field of view angle adjustment value, the second field of view angle is adjusted within a preset angle range.

9. The method according to any one of claims 1-8, characterized in that, Before determining the field of view region of the VR image based on the first field of view angle and the predicted viewpoint position of the wearer, the method further includes: Based on the wearer's current true viewpoint position and / or region center offset value, the wearer's viewpoint position is predicted to obtain the wearer's predicted viewpoint position.

10. The method according to any one of claims 1-8, characterized in that, After determining the field of view of the VR image, the method further includes: A VR data stream is generated based on the image content of the VR image located within the field of view, and the VR data stream is sent to the VR device for the VR device to display the image content.

11. A device for determining a field of view, characterized in that, include: The data acquisition module is used to acquire display latency data of the virtual reality (VR) device and acquire the current first head rotation data of the user wearing the VR device. An angle determination module is used to determine the first field of view angle of the VR image based on the display delay data and the first head rotation data; The region determination module is used to determine the field of view region of the VR image based on the first field of view angle and the predicted viewpoint position of the wearer.

12. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the method for determining the field of view region according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for determining the field of view region as described in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method for determining the field of view region as described in any one of claims 1-10.

Images