Virtual reality display apparatus, and control method therefor

Abstract

The present application belongs to the technical field of display. Disclosed are a virtual reality display apparatus, and a control method for a virtual reality display apparatus. The virtual reality display apparatus comprises a control component, a bracket, a display unit, a first sensing component and a second sensing component. An eye movement speed and a head movement speed of a user who wears the bracket, a rendering frame rate, and a hardware resource occupancy rate of the virtual reality display apparatus are acquired, and by means of a control component, a display resolution of the display unit is adjusted on the basis of at least one parameter of the eye movement speed, the head movement speed, the rendering frame rate and the hardware resource occpancy rate, such that the virtual reality display apparatus can perform display at different resolutions on the basis of different conditions. Therefore, the problem in the relevant art of the flexibility of a virtual reality display apparatus being poor during display is solved, and the effect of improving the flexibility of the virtual reality display apparatus during display is achieved.

Description

Virtual reality display device and control method of virtual reality display device Technical Field

[0001] The present application relates to the field of display technology, and in particular to a virtual reality display device and a control method for the virtual reality display device. Background Art

[0002] A virtual reality display device is a device that can realize display functions.

[0003] A current virtual reality display device includes a control component, a helmet, and a display unit. The control component and the display unit are installed on the helmet. When a user wears the helmet, the display unit displays a display image to the user to realize the display function.

[0004] However, the above virtual reality display device has poor flexibility during display.

[0005] Summary of the Invention

[0006] The present invention provides a virtual reality display device and a method for controlling the virtual reality display device. The technical solution is as follows:

[0007] According to a first aspect of the present application, a virtual reality display device is provided, comprising: a control component, a bracket, a display unit, a first sensor component, and a second sensor component, wherein the display unit is located on the bracket;

[0008] The first sensing component is used to obtain eye movement data of a user wearing the bracket;

[0009] The second sensing component is used to obtain head movement data of a user wearing the support;

[0010] The control component is used to determine the eye movement velocity based on the eye movement data, and determine the head movement velocity based on the head movement data;

[0011] The control component is used to obtain a rendering frame rate of a display image of the display unit and a hardware resource occupancy rate of the virtual reality display device;

[0012] The control component is further configured to adjust the display resolution of the display unit based on at least one parameter among the eye movement velocity, the head movement velocity, the rendering frame rate, and the hardware resource occupancy rate.

[0013] Optionally, the virtual reality display device further includes a power supply component;

[0014] The control component is further configured to obtain the power level of the power supply component, the scene type of the display scene of the display unit, and the model complexity in the display screen;

[0015] The control component is used to adjust the display resolution of the display unit based on at least one parameter among the eye movement speed, the head movement speed, the rendering frame rate, the hardware resource occupancy rate, and at least one parameter among the power level, the scene type and the model complexity.

[0016] Optionally, the display resolution of the display unit has multiple resolution levels, and the size of the resolution level is positively correlated with the size of the display resolution of the display unit;

[0017] The control component is used to:

[0018] determining a comprehensive parameter based on the eye movement speed, the head movement speed, the rendering frame rate, the power, the hardware resource occupancy rate, and the model complexity, wherein the eye movement speed, the head movement speed, the rendering frame rate, and the power are negatively correlated with the comprehensive parameter, and the model complexity is positively correlated with the comprehensive parameter;

[0019] In response to the comprehensive parameter being greater than a specified value, reducing a resolution level of the display unit;

[0020] In response to the comprehensive parameter being less than or equal to a specified value, the resolution level of the display unit is increased.

[0021] Optionally, the control component is used to:

[0022] The comprehensive parameter is determined by a preset formula, wherein the preset formula includes: y=h+e+t+x*f+b+c+g;

[0023] Among them, y is the comprehensive parameter, h is the normalized parameter of the head movement speed, e is the normalized parameter of the eye movement speed, t is the normalized parameter of the model complexity, f is the normalized parameter of the rendering frame rate, x is the weight of the rendering frame rate, b is the normalized parameter of the power, and the hardware resource occupancy rate includes c and g, c is the occupancy rate of the processor in the control component, and g is the occupancy rate of the graphics processor in the control component.

[0024] Optionally, the first sensing component includes a gaze point sensing unit, and the control component is configured to:

[0025] In response to the scene type being a planar scene, turning off the gaze point sensing unit;

[0026] In response to the power level being greater than the target power level, controlling the resolution of the display unit to be a first resolution;

[0027] In response to the power level being less than or equal to the target power level, the resolution of the display unit is controlled to be a second resolution, where the second resolution is smaller than the first resolution.

[0028] Optionally, the control component is used to:

[0029] In response to the scene type being a stereoscopic scene, determining whether the head movement speed is greater than a target head movement speed;

[0030] In response to the head movement speed being greater than the target head movement speed, turning off the gaze point sensing unit;

[0031] In response to the head movement speed being not greater than the target head movement speed, turning on the gaze point sensing unit.

[0032] Optionally, the control component is used to:

[0033] determining the eye movement velocity based on data collected by the gaze point sensing unit;

[0034] In response to the eye movement velocity being less than a target eye movement velocity, determining a gaze area of ​​the user in a display area of ​​the display unit based on the gaze point sensing unit;

[0035] The pixel density of the gaze area in the display area is controlled to be a first pixel density, and the pixel density of an area other than the gaze area in the display area is controlled to be a second pixel density, wherein the first pixel density is greater than the second pixel density.

[0036] Optionally, the control component is used to:

[0037] In response to the eye movement velocity being greater than the target eye movement velocity, controlling the gaze point sensing unit to detect the eye movement velocity of the user at a first shutter speed;

[0038] In response to the eye movement velocity being less than or equal to the target eye movement velocity, the gaze point sensing unit is controlled to detect the eye movement velocity of the user at a second shutter speed, the second shutter speed being less than the first shutter speed.

[0039] Optionally, the control component is used to:

[0040] In response to the scene complexity being in a first interval, setting a ratio of the gaze area to the display area to a first ratio value;

[0041] In response to the scene complexity being in a second interval, setting the proportion of the gaze area to the display area to a second proportion value, wherein a minimum value of the second interval is greater than a maximum value of the first interval, and the second proportion value is less than the first proportion value;

[0042] In response to the scene complexity being in a third interval, the proportion of the gaze area to the display area is set to a third proportion value, the minimum value of the third interval is greater than the maximum value of the second interval, and the third proportion value is less than the second proportion value.

[0043] Optionally, the control component is used to:

[0044] When the ratio of the gaze area to the display area is the first ratio value, determining whether the rendering frame rate is less than a target frame rate;

[0045] In response to the rendering frame rate being less than the target frame rate, adjusting the size of the gaze area so that the proportion of the gaze area to the display area is the second proportion value;

[0046] When the ratio of the gaze area to the display area is the second ratio value, determining whether the rendering frame rate is less than a target frame rate;

[0047] In response to the rendering frame rate being lower than the target frame rate, the size of the gaze area is adjusted so that the proportion of the gaze area to the display area is the third proportion value.

[0048] Optionally, the control component is used to:

[0049] Setting a vertex number acquisition program on each three-dimensional object in the display scene;

[0050] Obtaining the number of vertices of a three-dimensional object of the display scene located in the display screen;

[0051] The model complexity is determined based on the number of vertices.

[0052] Optionally, the control component is used to:

[0053] Acquiring acceleration data and gyroscope data of the bracket within a specified time period through the second sensing component;

[0054] Obtaining a quaternion based on the specified duration, the acceleration data, and the gyroscope data;

[0055] The head movement velocity is determined based on the quaternion.

[0056] Optionally, the control component is used to:

[0057] Obtaining a name of display data corresponding to the display screen, wherein the name includes data of the scene type;

[0058] The scene type is determined based on the name.

[0059] According to another aspect of an embodiment of the present application, a method for controlling a virtual reality display device is provided. The virtual reality display device includes: a control component, a bracket, a display unit, a first sensor component, and a second sensor. The display unit is located on the bracket. The method includes:

[0060] Acquiring eye movement data of a user wearing the bracket through the first sensing component;

[0061] acquiring head movement data of a user wearing the support through the second sensing component;

[0062] determining an eye movement velocity based on the eye movement data, and determining a head movement velocity based on the head movement data;

[0063] Obtaining a rendering frame rate of a display image of the display unit and a hardware resource occupancy rate of the virtual reality display device;

[0064] The display resolution of the display unit is adjusted based on at least one parameter among the eye movement velocity, the head movement velocity, the rendering frame rate, and the hardware resource occupancy rate.

[0065] According to another aspect of an embodiment of the present application, a virtual reality display device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the control method of the virtual reality display device as described above.

[0066] According to another aspect of an embodiment of the present application, a virtual reality display device is provided, comprising: a control component, a bracket, a display unit, a first sensor component, and a second sensor component, wherein the display unit is located on the bracket;

[0067] The control component controls the display unit to display;

[0068] The first sensor acquires eye movement data of a user wearing the support and provides the eye movement data to the control component;

[0069] The second sensing component acquires head movement data of the user wearing the support and provides the head movement data to the control component;

[0070] The control component determines an eye movement velocity based on the eye movement data, and determines a head movement velocity based on the head movement data;

[0071] The control component obtains a rendering frame rate of a display image of the display unit and a hardware resource occupancy rate of the virtual reality display device;

[0072] The control component adjusts the display resolution of the display unit based on at least one parameter among the eye movement velocity, the head movement velocity, the rendering frame rate, and the hardware resource occupancy rate.

[0073] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0074] Provided is a virtual reality display device comprising a control component, a bracket, a display unit, and a sensor component. By acquiring the eye movement velocity, head movement velocity, rendering frame rate, and hardware resource occupancy rate of a user wearing the bracket, and adjusting the display resolution of the display unit based on at least one parameter among the eye movement velocity, head movement velocity, rendering frame rate, and hardware resource occupancy rate, the control component enables the virtual reality display device to display at different resolutions based on different situations. This solves the problem of poor display flexibility of virtual reality display devices in related technologies, thereby enhancing the display flexibility of the virtual reality display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0076] FIG1 is a structural block diagram of a virtual reality display device provided in an embodiment of the present application;

[0077] FIG2 is a structural block diagram of another virtual reality display device provided in an embodiment of the present application;

[0078] FIG3 is a flow chart of a control component for determining head movement speed according to an embodiment of the present application;

[0079] FIG4 is a flow chart of a method for determining model complexity provided by an embodiment of the present application;

[0080] FIG5 is a flowchart of determining a rendering frame rate in a virtual reality device provided in an embodiment of the present application;

[0081] FIG6 is a flowchart of a software process in a virtual reality display device provided in an embodiment of the present application;

[0082] FIG7 is a flow chart of an overall discrimination mechanism in a virtual reality display device provided in an embodiment of the present application;

[0083] FIG8 is a flowchart of a display method of a virtual reality display device provided in an embodiment of the present application;

[0084] FIG9 is a flowchart of another display method of a virtual reality display device provided in an embodiment of the present application.

[0085] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0086] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0087] A virtual reality device is a display device that combines virtuality and reality and can provide users with an immersive display effect.

[0088] A virtual reality device may include a processor, a display, a sensor component and a helmet. The processor, display and sensor component are all installed on the helmet. After the user wears the helmet, the sensor component can obtain the user's head posture. The processor can generate image data based on the head posture and control the display to display based on the image data.

[0089] However, this display method has poor flexibility.

[0090] In addition, the above-mentioned virtual reality device may consume a large amount of power during display.

[0091] The embodiments of the present application provide a virtual reality display device and a method for controlling the virtual reality display device, which can solve the problems existing in the above-mentioned related technologies.

[0092] It should be noted that the virtual reality device involved in the embodiments of the present application does not refer to a device that only applies virtual reality (VR) technology. The device can also apply augmented reality (AR) technology and mixed reality (MR) technology.

[0093] Figure 1 is a structural block diagram of a virtual reality display device provided in an embodiment of the present application. The virtual reality display device includes: a control component 11, a bracket 12, a display unit 13, a first sensor component 141 and a second sensor 142, and the display unit 13 is located on the bracket 12.

[0094] The first sensing component 141 is used to obtain eye movement data of the user wearing the bracket 12.

[0095] The second sensor component 142 is used to obtain head movement data of the user wearing the bracket 12.

[0096] The control component 11 is used to determine the eye movement velocity based on the eye movement data, and to determine the head movement velocity based on the head movement data.

[0097] The control component 11 is used to obtain the rendering frame rate of the display image of the display unit and the hardware resource occupancy rate of the virtual reality display device.

[0098] The control component 11 is further configured to adjust the display resolution of the display unit based on at least one parameter selected from the group consisting of eye movement velocity, head movement velocity, rendering frame rate, and hardware resource occupancy rate.

[0099] In summary, the embodiments of the present application provide a virtual reality display device including a control component, a bracket, a display unit, and a sensing component. By obtaining the eye movement speed, head movement speed, rendering frame rate, and hardware resource occupancy rate of a user wearing the bracket, and adjusting the display resolution of the display unit based on at least one parameter among the eye movement speed, head movement speed, rendering frame rate, and hardware resource occupancy rate through the control component, the virtual reality display device can display with different resolutions based on different situations, thereby solving the problem of poor flexibility of the virtual reality display device during display in the related art and achieving the effect of improving the flexibility of the virtual reality display device during display.

[0100] Please refer to FIG. 1 , where the control component 11 controls the display unit 13 to display.

[0101] The first sensor 141 acquires eye movement data of the user wearing the support 12 and provides the eye movement data to the control component 11 .

[0102] The second sensing component 142 obtains head movement data of the user wearing the support 12 and provides the head movement data to the control component 11.

[0103] The control component 11 determines the eye movement velocity based on the eye movement data, and determines the head movement velocity based on the head movement data.

[0104] The control component 11 obtains the rendering frame rate of the display image of the display unit 13 and the hardware resource occupancy rate of the virtual reality display device.

[0105] The control component 11 adjusts the display resolution of the display unit 13 based on at least one parameter of eye movement velocity, head movement velocity, rendering frame rate, and hardware resource occupancy rate.

[0106] Figure 2 is a structural block diagram of another virtual reality display device provided in an embodiment of the present application. The virtual reality display device includes: a control component 11, a bracket 12, a display unit 13, a first sensor component 141, a second sensor component 142 and a power supply component 15, and the display unit 13 is located on the bracket 12.

[0107] The control component 11 is further used to obtain the power level of the power supply component 15, the scene type of the display scene of the display unit 13, and the model complexity in the display screen.

[0108] The control component 11 is used to adjust the display resolution of the display unit 13 based on at least one parameter of eye movement speed, head movement speed, rendering frame rate, hardware resource occupancy rate, and at least one parameter of power, scene type and model complexity.

[0109] It should be noted that the display unit 13 may have an actual physical resolution, which may be determined by the number of pixels in the display area of ​​the display unit 13. For example, if the display area of ​​the display unit 13 may include 1000×1000 pixel units, the physical resolution of the display unit 13 may be 1000×1000. The display resolution may refer to the resolution of the image displayed by the display unit 13. The display resolution is less than or equal to the physical resolution. The control component 11 also adjusts the display resolution of the display unit 13. When adjusting, the display resolution can be adjusted by having multiple pixel units display one pixel in the image. For example, if the physical resolution of the display unit 13 is 1000×1000, then the display unit 13 has pixel units arranged in 1000 rows and 1000 columns. When the display resolution of the display unit is adjusted to 100×100, this can be achieved by having every 10 pixel units display one pixel. In this way, the display resolution of the display unit 13 can be adjusted.

[0110] The control component 11 can adjust the display resolution of the display unit 13 within a range that is less than or equal to the physical resolution of the display unit 13 .

[0111] In addition, the virtual reality display device may further include a communication module 16, a storage module 17, an audio module 18, an indicator light module 19 and an input module 20. These modules may implement various functions under the control of the control module.

[0112] In an exemplary embodiment, the display resolution of the display unit 13 may have multiple resolution levels, and the size of the resolution level is positively correlated with the size of the display resolution of the display unit, that is, the higher the resolution level, the higher the display resolution of the display unit.

[0113] For example, the display unit 13 may have five resolution levels. The display resolution at resolution level 1 may be 0.2p1×0.2p2, where p1 is the horizontal physical resolution of the display unit 13 and p2 is the vertical physical resolution of the display unit 13. The display resolution at resolution level 2 may be 0.4p1×0.4p2, the display resolution at resolution level 3 may be 0.6p1×0.6p2, the display resolution at resolution level 4 may be 0.8p1×0.8p2, and the display resolution at resolution level 5 may be p1×p2. Of course, the display unit 13 may also have other numbers of resolution levels and the division method of each level, which is not limited in the embodiments of the present application.

[0114] In this case, the control component 11 can be used to:

[0115] Comprehensive parameters are determined based on eye movement speed, head movement speed, rendering frame rate, hardware resource occupancy rate and model complexity. Among them, eye movement speed, head movement speed, rendering frame rate and power are negatively correlated with comprehensive parameters, while model complexity is positively correlated with comprehensive parameters.

[0116] In response to the comprehensive parameter being greater than a specified value, the resolution level of the display unit 13 is lowered.

[0117] In response to the comprehensive parameter being less than or equal to the specified value, the resolution level of the display unit 13 is increased.

[0118] The comprehensive parameter determined based on eye movement velocity, head movement velocity, rendering frame rate, hardware resource utilization, and model complexity can, to a certain extent, reflect the current system load of the virtual reality display device. When the comprehensive parameter is large (e.g., greater than a specified value), the system load is large, and the resolution level of the display unit 13 can be reduced to alleviate the system load, reduce power consumption, and increase the battery life of the virtual reality display device. When the comprehensive parameter is small (e.g., less than or equal to a specified value), the system load is small, and the resolution level of the display unit 13 can be increased to improve the display quality of the virtual reality display device and enhance the user experience.

[0119] Among them, the control component can obtain the hardware resource occupancy rate by calling a preset node. Exemplarily, the hardware resource occupancy rate includes the processor occupancy rate and the graphics processor occupancy rate. Correspondingly, the system can include a first node and a second node. The control component can call the first node to obtain the processor occupancy rate, and obtain the graphics processor occupancy rate by calling the second node.

[0120] The virtual reality display device provided in the embodiment of the present application can obtain head movement velocity in a variety of ways. For example, please refer to FIG3 , which is a flow chart of a control component provided in the embodiment of the present application for determining head movement velocity. The control component can be used to perform the following steps:

[0121] Step 301: Acquire acceleration data and gyroscope data of the bracket during a specified time period through a second sensor component.

[0122] The second sensing component may include an inertial measurement unit (IMU). The control component may obtain head movement data collected by the inertial sensor, including acceleration data and gyroscope data of the bracket in a specified period of time. The bracket may be a bracket worn by the user when using the virtual reality device. Exemplarily, the bracket may include various brackets such as a helmet-type bracket and a glasses-type bracket. The acceleration data and gyroscope data of the bracket may reflect the movement of the user's head.

[0123] Step 302: Obtain a quaternion based on the specified duration, acceleration data, and gyroscope data.

[0124] The control component can fuse quaternions based on the specified duration, acceleration data, and gyroscope data.

[0125] Step 303: Determine the head movement velocity based on the quaternion.

[0126] The control component may obtain a gyroscope synthetic linear velocity based on the quaternion determined in step 302 , and then convert the gyroscope synthetic linear velocity into an angular velocity, which may be a head movement velocity.

[0127] That is, the control component 11 can be used to:

[0128] The acceleration data and gyroscope data of the bracket in a specified time period are obtained through the second sensor component.

[0129] Gets a quaternion based on the specified duration, acceleration data, and gyroscope data.

[0130] Determine head velocity based on quaternions.

[0131] The virtual reality display device provided in the embodiments of the present application can determine the scene type in a variety of ways. In one way, the control component is used to:

[0132] Get the name of the display data corresponding to the display screen, which includes the scene type data.

[0133] Determines the scene type based on the name.

[0134] Scene types can include flat display scenes, stereoscopic display scenes, and game display scenes. In this approach, the control component can determine the scene type based on the name of the received display data. For example, it can be pre-agreed with the provider of the display data that the name of the display data includes the scene type data. For example, the application package name of the flat scene display data can be xxx.xxx.2Dvideo, the application package name of the panoramic video playback application can be xxx.xxx.3Dvideo, and the application package name of the game application can be xxx.xxx.game.

[0135] Of course, the control component may also detect the received display data to confirm the scene type, and the embodiment of the present application does not limit this.

[0136] The virtual reality display device provided in the embodiment of the present application can determine the model complexity in the display scene in a variety of ways. Optionally, the control component is used to:

[0137] Set up vertex count fetching routines on each solid object in the displayed scene.

[0138] Gets the number of vertices of the three-dimensional object in the display scene.

[0139] Determines the model complexity based on the number of vertices.

[0140] This method is a method for determining model complexity based on the number of fixed points of a three-dimensional object. Specifically, please refer to Figure 4, which is a flow chart of a method for determining model complexity provided by an embodiment of the present application. In this method, the control component can perform the following steps:

[0141] 401. Traverse all three-dimensional objects in the display scene and determine whether the vertex calculation script (vertex number acquisition program) is mounted on all three-dimensional objects. If it is mounted, exit this loop and continue traversal. If it is not mounted, add the vertex calculation script to the three-dimensional object.

[0142] 402. Start rendering monitoring.

[0143] In this step, the control component can use the delegate event Action to start rendering listening, add a rendering listening event to the Action, and then add a listener (Listener) to the rendering listening event. In this way, the rendering listening event is successfully started.

[0144] 403. Determine whether the mounted three-dimensional object is within the camera range of the virtual reality device.

[0145] The rendering listening event can determine in real time whether the mounted three-dimensional object is within the camera range of the virtual reality device in each frame. If it is within the camera range, the return value is true. When the return value is true, step 404 is executed. When the return value is false, there is no need to calculate the number of vertices of the three-dimensional object and continue to judge, so as to save system overhead.

[0146] 404. Determine the number of vertices of the three-dimensional object. Execute step 405.

[0147] When a stereo object is within the camera range, the control component can calculate the number of vertices of the stereo object.

[0148] 405. Count the sum of the vertex numbers of all objects within the camera range, and determine the model complexity based on the sum of the vertex numbers.

[0149] The control component can obtain the number of vertices of each object within the camera range, determine the sum of the number of vertices of these objects, and determine the model complexity based on the sum of the number of vertices of the objects.

[0150] In addition, the virtual reality display device provided in the embodiment of the present application can determine the eye movement speed of the user wearing the bracket in the virtual reality device in a variety of ways.

[0151] In an exemplary embodiment, the control component is configured to:

[0152] 1) Obtain the gaze point coordinates t1 and t2 of the human eye at two moments respectively.

[0153] The control component can determine the gaze point coordinates t1 and t2 of the human eye at two moments based on the data provided by the first sensing component, wherein the first sensing component may include a gaze point sensing unit, and the gaze point sensing unit may include an image acquisition subunit (such as a camera, etc.), and the image acquisition subunit may acquire eye movement data including image data of the human eye, and provide the eye movement data to the control component so that the control component can determine the gaze point coordinates of the human eye based on the image data.

[0154] 2) Obtain the eye's line of sight rotation angle θ between two moments.

[0155] The control component can determine the sight rotation angle θ of the human eye between two moments based on the data provided by the first sensing component.

[0156] 3) Determine the eye movement velocity based on the gaze point coordinates at two moments and the achieved rotation angle.

[0157] Exemplarily, eye velocity = θ(t2-t1).

[0158] Of course, the virtual reality display device provided in the embodiment of the present application can also determine the eye movement speed through other methods, and the embodiment of the present application is not limited to this.

[0159] In addition, the virtual reality display device provided in the embodiment of the present application can determine the rendering frame rate of the display image of the display unit in a variety of ways. Optionally, please refer to Figure 5, which is a flowchart of determining the rendering frame rate in the virtual reality device provided in the embodiment of the present application. The control component is used to perform the following steps:

[0160] Step 501: Determine the time interval and record the current first moment.

[0161] The time interval is a period for determining the rendering frame rate. The control component can determine the rendering frame rate in the time interval to avoid fluctuations in the rendering frame rate that affect detection accuracy. For example, the time interval can be 0.5 seconds.

[0162] Step 502: When each frame is displayed, the recorded number of rendered frames is increased by 1, and it is determined whether the difference between the current second moment and the first moment reaches the time interval.

[0163] If the difference between the current second moment and the first moment and the first moment reaches the time interval, step 503 is executed. If the difference between the current second moment and the first moment and the first moment does not reach the time interval, step 502 can be continued and re-executed.

[0164] Exemplarily, the control component may increase the number of recorded rendering frames by 1 when a vertical synchronization signal (V-sync) is obtained.

[0165] Step 503: Determine the rendering frame rate.

[0166] If the difference between the current second moment and the first moment reaches the time interval, the control component can determine the rendering frame rate = F / (t2-t1), where F is the currently recorded rendering frame number, t2 is the second moment, and t1 is the first moment.

[0167] In the embodiment of the present application, the control component 11 may determine the comprehensive parameters in a variety of ways. For example, the control component 11 is used to:

[0168] The comprehensive parameters are determined by a preset formula, which includes: y=h+e+t+x*f+b+c+g;

[0169] Among them, y is a comprehensive parameter, h is a normalized parameter of head movement velocity, e is a normalized parameter of eye movement velocity, t is a normalized parameter of model complexity, f is a normalized parameter of rendering frame rate, x is the weight of rendering frame rate (for example, x can be 2), b is a normalized parameter of power, and the hardware resource occupancy rate includes c and g, c is the occupancy rate of the processor in the control component, and g is the occupancy rate of the graphics processor in the control component.

[0170] In the embodiment of the present application, the control component can normalize the above parameters in a variety of ways, for example:

[0171] h = 1 - n1 / 30, (0 ≤ n1 ≤ 30), where n1 is the head movement speed, which can be expressed in degrees per second. The unit can be removed based on normalization.

[0172] e = 1 - n2 / 30, (0 ≤ n1 ≤ 30), where n2 is the eye movement velocity, which can be expressed in degrees per second. Here, the unit can be removed based on normalization.

[0173] t=1, (0.5q≤t≤q), where q is a threshold of scene complexity, which may be determined by the total number of vertices of three-dimensional objects in the rendered image; for example, it may be 1 million.

[0174] t=0.5,(0.1q≤t<0.5q);

[0175] t=0.2,(0.01q≤t<0.1q);

[0176] t=0.1,(0≤t<0.01q);

[0177] f = 1 - n4 / 90, (0 ≤ n4 ≤ 90), where n4 is the rendering frame rate. The 90 in the calculation formula varies according to the system frame rate setting. Here, we take 90 frames as an example. If the system frame rate is 75 or 60, the formula will also change accordingly;

[0178] b=1,(0%≤n5<20%),n5 is the amount of electricity;

[0179] b=0.5, (20%≤n5<50%);

[0180] b=0, (50%≤n5≤100%);

[0181] 0%≤c≤100%, where c is the CPU usage;

[0182] 0%≤g≤100%, where x is the graphics processing unit (GPU) occupancy rate.

[0183] According to the above calculation method, the comprehensive coefficient y is calculated in real time for each frame, 0≤y≤8. When y is closer to 8, it means that the system performance is close to the limit. Increasing the system burden will affect the frame rate and experience. Therefore, when y>7, the rendering resolution level can be reduced. When y<4, if the rendering resolution does not reach the highest level, the resolution level can be increased to achieve the effect of dynamically adjusting the system rendering resolution in real time.

[0184] Optionally, the first sensing component 141 includes a gaze point sensing unit, which can be used to obtain the user's gaze point and the user's eye movement speed, wherein the user's gaze point can be combined with the foveated rendering technology to achieve the effect of reducing the system load. Among them, the foveated rendering technology is a technology that can be applied to virtual reality devices. This technology can perform high-resolution rendering in the area where the user's gaze point is located, and perform lower-resolution rendering around the area where the gaze point is located. Because when the human eye sees something, the entire field of view is not equally clear, but the center point is clear and becomes blurrier towards the sides. Therefore, the foveated rendering technology can reduce the system load without affecting the user experience.

[0185] The control component 11 is used to:

[0186] In response to the scene type being a planar scene, the gaze point sensing unit 141 is turned off. A planar scene may refer to a planar image or a fixed-viewing angle stereoscopic scene displayed by the display unit (since such scenes do not involve a change in viewing angle, they can be considered planar scenes). Such scenes do not involve rendering of three-dimensional objects, so the computational complexity of rendering a planar scene is relatively small, and the load pressure on the system is relatively small. In this case, the gaze point sensing unit 141 may be turned off, and the gaze point rendering technology is not used. The entire display area of ​​the display unit 13 is set to a higher resolution level. The gaze point sensing unit 141 may include devices such as cameras to collect parameters such as images of the human eye and obtain gaze point data of the human eye.

[0187] In response to the power level being greater than the target power level, the resolution of the display unit is controlled to be the first resolution.

[0188] In response to the power level being less than or equal to the target power level, the resolution of the display unit is controlled to be a second resolution, which is smaller than the first resolution.

[0189] The first resolution is a larger resolution than the second resolution. This is because when the power level is greater than the target power level, the power level of the virtual reality device is sufficient, and because the scene type of the current display unit is a flat scene, the control component can set the resolution of the entire display area of ​​the display unit 13 to a larger resolution. For example, the resolution of the entire display area of ​​the display unit 13 can be set to the same resolution as the physical resolution, or to a resolution level that is one level lower than the highest resolution level. For example, the first resolution can be 2664×2880, and the second resolution can be 2368×2560.

[0190] In addition, the control component 11 may include a processor and a graphics processor. When adjusting the resolution level, the processor can adjust the instructions issued to the graphics processor. The graphics processor can render the image at the adjusted resolution level based on the adjusted instructions, and transmit the data of the rendered image to the processor, which then provides it to the display unit for display.

[0191] Optionally, the control component 11 is used to:

[0192] In response to the scene type being a stereoscopic scene, a determination is made as to whether the head movement speed is greater than a target head movement speed. In a stereoscopic scene, the control component 11 may begin determining whether to enable the foveated rendering function. Since it may be difficult to clearly focus on a certain area when the overall eye movement speed is fast, the foveated rendering function may be enabled when the overall eye movement speed is slow. The overall eye movement speed can be determined by the head movement speed and the eye movement speed. The control component may first determine whether the head movement speed is greater than the target head movement speed. For example, the target head movement speed may be 30 degrees per second.

[0193] In response to the head movement speed being greater than the target head movement speed, the gaze point sensing unit 141 is turned off. When the head movement speed is large, the control component 11 can turn off the gaze point sensing unit 141. If the gaze point sensing unit 141 is currently in the off state, the off state of the gaze point sensing unit 141 can be maintained without adjustment.

[0194] In response to the head movement speed not being greater than the target head movement speed, the gaze point sensing unit 141 is turned on. When the head movement speed is greater, the control component 11 may turn on the gaze point sensing unit 141.

[0195] After turning on the gaze point sensing unit 141, the control component can further make the next judgment based on the eye movement speed.

[0196] In an exemplary embodiment, the control component 11 is configured to:

[0197] The eye movement velocity is determined based on the data collected by the gaze point sensing unit 141.

[0198] In response to the eye movement velocity being less than the target eye movement velocity, the user's gaze area in the display area of ​​the display unit is determined based on the gaze point sensing unit 141. When the eye movement velocity is less than the target eye movement velocity, the control component 11 can start the gaze point rendering function and determine the user's gaze area in the display area of ​​the display unit based on the data collected by the gaze point sensing unit 141 (such as the gaze point of the human eye in the display scene). The gaze area can be an area including the gaze point of the human eye. The size of the gaze area can be pre-set and adjusted based on some parameters. Among them, the target eye movement velocity can be 120 degrees / second.

[0199] The pixel density of the gaze area in the display area is controlled to be a first pixel density, and the pixel density of the area other than the gaze area in the display area is controlled to be a second pixel density, and the first pixel density is greater than the second pixel density. Pixel density can refer to the number of pixels per inch. The higher the pixel density, the clearer and finer the picture. However, correspondingly, the greater the pixel density, the larger the pixel data will be in the same size, and the resolution will also be greater, and the load on the system will also increase. Therefore, the control component can increase the pixel density of the area where the user is looking at, so that the user's gaze area is a "high-definition area", and reduce the pixel density of the area not looked at by the human eye, so that other areas not looked at by the user are "low-definition areas", so as to reduce the system load without affecting the user experience and increase the usage time of the virtual reality device.

[0200] In response to the eye movement velocity being greater than the target eye movement velocity, the gaze point sensing unit 141 is controlled to detect the user's eye movement velocity at a first shutter speed. When the eye movement velocity is greater than the target eye movement velocity, it indicates that the human eye is in a "jumping" state, and it is difficult for the human eye to clearly focus on a certain area. Therefore, there is no need to enable the gaze point rendering function. At this time, the control component can control the gaze point sensing unit 141 to detect the user's eye movement velocity at a higher first shutter speed to accurately obtain the eye movement velocity. In addition, the control component 11 can control the entire display area of ​​the display unit to render at a lower resolution, for example, it can be rendered at the lowest resolution level, or at a resolution level one level higher than the lowest resolution level, etc.

[0201] In response to the eye movement velocity being less than the target eye movement velocity, the gaze point sensing unit 141 is controlled to detect the eye movement velocity of the user at a second shutter speed that is less than the first shutter speed.

[0202] When the eye movement speed is less than the target eye movement speed, it indicates that the current eye movement speed is slow. At this time, the control component can control the gaze point sensing unit 141 to detect the user's eye movement speed at a slower second shutter speed.

[0203] When the eye movement velocity is less than the target eye movement velocity, the control component may further perform corresponding control based on the scene complexity. Optionally, the control component 11 is configured to:

[0204] In response to the scene complexity being in a first interval, the ratio of the gaze area to the display area is set to a first ratio value.

[0205] In response to the scene complexity being in a second interval, the ratio of the gaze area to the display area is set to a second ratio value, the minimum value of the second interval is greater than the maximum value of the first interval, and the second ratio value is less than the first ratio value.

[0206] In response to the scene complexity being in a third interval, the ratio of the gaze area to the display area is set to a third ratio value, the minimum value of the third interval is greater than the maximum value of the second interval, and the third ratio value is less than the second ratio value.

[0207] Among them, the gaze area is a high-definition area with a higher resolution than other areas in the display area of ​​the display unit. The larger the proportion of the gaze area to the display area, the greater the load on the system. Therefore, the control component can adjust the proportion of the gaze area to the display area based on the scene complexity to balance the system load under various scene complexities, improve the battery life of the virtual reality device, and thus achieve the effect of improving user experience.

[0208] The first interval, the second interval and the third interval can be determined based on the threshold of scene complexity. For example, the first interval can be 0≤t<0.1q, where q is the threshold of scene complexity, and the scene complexity can be determined by the total number of vertices of three-dimensional objects in the rendered picture; the second interval can be 0.1q≤t<0.5q; and the third interval can be 0.5q≤t≤q.

[0209] In addition, when adjusting the size of the gaze area, the pixel density of the gaze area can also be adjusted accordingly. For example, the pixel density of the gaze area can be reduced as the proportion of the gaze area in the display area is reduced, or the pixel density of the gaze area can remain unchanged when the proportion of the gaze area in the display area is reduced.

[0210] Optionally, the control component 11 is further configured to:

[0211] When the ratio of the gaze area to the display area is a first ratio value, determining whether the rendering frame rate is less than a target frame rate; the target frame rate may be a preset frame rate, at or above which the virtual reality device can achieve a smooth display effect. For example, the target frame rate may be 60 Hz to 240 Hz, such as 90 Hz, 120 Hz, etc.

[0212] In response to the rendering frame rate being less than the target frame rate, the size of the gaze area is adjusted so that the proportion of the gaze area to the display area is a second ratio. When the rendering frame rate is less than the target frame rate, indicating that the current system load is too high to ensure a smooth display, the control component can reduce the proportion of the gaze area to the display area from the first ratio to the second ratio to reduce the system load. This approach ensures the rendering frame rate by reducing the size of the gaze area, improving the user's viewing experience and avoiding any perceived lag or stuttering.

[0213] When the proportion of the gaze area to the display area is a second proportion value, determine whether the rendering frame rate is less than the target frame rate; after reducing the proportion of the gaze area to the display area, the control component can again determine whether the rendering frame rate is greater than the target frame rate to confirm whether the above operation achieves the effect of ensuring the rendering frame rate.

[0214] In response to the rendering frame rate being less than the target frame rate, the size of the gaze area is adjusted so that the proportion of the gaze area to the display area is a third ratio value. When the rendering frame rate still meets the target frame rate, the control component can further reduce the proportion of the gaze area to the display area from the second ratio value to the third ratio value.

[0215] In addition, the proportion of the gaze area to the display area can have a minimum proportion value, which can be the proportion of the area that can be clearly seen when the human eye is gazing at it in the display area. The control component can reduce the proportion of the gaze area to the display area in the above manner until it is reduced to the minimum proportion value.

[0216] Figure 6 is a software processing flow chart in the virtual reality display device provided by an embodiment of the present application. Please refer to Figure 6, wherein the information input module can be used to provide corresponding data to the eye movement speed and head movement speed calculation module. The eye movement speed and head movement speed calculation module can transmit the calculated eye movement speed and head movement speed to the real-time acquisition module. The scene content information module can obtain the scene information and input it into the real-time acquisition module after processing by the initialization module (the initialization module can have an information storage module for storing the complexity of each part of the scene. When the head is turned in real time or the scene is updated, the current complexity can be calculated through the stored information, so there is no need to calculate the complexity in real time.). The real-time acquisition module can correspondingly determine the data of the head posture information. The real-time acquisition module can transmit the determined data to the discrimination mechanism module, and the discrimination mechanism module judges the received data and transmits the judgment result to the foveated rendering module. The foveated rendering module can cache the previous frame image information based on the rendering task, and process it in sequence through the ATW (asynchronous time warp) processing module, the SDK rendering module, and the DRM video memory rendering module, and pass the image data to the driving module, which drives the first display screen and the second display screen to display (one of the first display screen and the second display screen is a left-eye display screen, and the other is a left-eye display screen).

[0217] FIG7 is a flowchart of an overall discrimination mechanism in a virtual reality display device provided by an embodiment of the present application, wherein the SDK, DLL, SDKJAR, application, and script may be programs in the virtual reality display device, and these programs may have different roles and functions in different locations. For example, the head velocity module can calculate head velocity based on data obtained from the inertial sensor IMU by the connected SDK. The eye velocity module can calculate eye velocity based on data obtained from the connected DLL. The head velocity module and the module connected to the right of the eye velocity module can disable the gaze point function (e.g., disable the gaze point sensing unit) when the head velocity module and / or the eye velocity exceed a threshold. The scene type module can determine the scene type based on the corresponding connected application and determine different control information for different scenarios based on the module connected to the right. After these modules determine the control information, they can write this control information into the hardware based on the SDK connected to the hardware on the right, so that the virtual reality display device can achieve the corresponding control effect.

[0218] In summary, the embodiments of the present application provide a virtual reality display device including a control component, a bracket, a display unit, and a sensing component. By obtaining the eye movement speed, head movement speed, rendering frame rate, and hardware resource occupancy rate of a user wearing the bracket, and adjusting the display resolution of the display unit based on at least one parameter among the eye movement speed, head movement speed, rendering frame rate, and hardware resource occupancy rate through the control component, the virtual reality display device can display with different resolutions based on different situations, thereby solving the problem of poor flexibility of the virtual reality display device during display in the related art and achieving the effect of improving the flexibility of the virtual reality display device during display.

[0219] FIG8 is a flowchart of a display method for a virtual reality display device provided in an embodiment of the present application. The method can be applied to the virtual reality display device provided in the above embodiment. The method includes the following steps:

[0220] Step 801: Acquire eye movement data of a user wearing the bracket through a first sensing component.

[0221] Step 802: Acquire head movement data of the user wearing the bracket through the second sensor component.

[0222] Step 803: Determine eye movement velocity based on the eye movement data, and determine head movement velocity based on the head movement data.

[0223] Step 804: Obtain the rendering frame rate of the display image of the display unit and the hardware resource occupancy rate of the virtual reality display device;

[0224] Step 805: Adjust the display resolution of the display unit based on at least one parameter of eye movement speed, head movement speed, rendering frame rate, and hardware resource occupancy rate.

[0225] In summary, the embodiments of the present application provide a method for controlling a virtual reality display device, by obtaining the eye movement speed, head movement speed, rendering frame rate of the user wearing the bracket and the hardware resource occupancy rate of the virtual reality display device, and adjusting the display resolution of the display unit based on at least one parameter among the eye movement speed, head movement speed, rendering frame rate and hardware resource occupancy rate through a control component, so that the virtual reality display device can display with different resolutions based on different situations, thereby solving the problem of poor flexibility of the virtual reality display device during display in the related art and achieving the effect of improving the flexibility of the virtual reality display device during display.

[0226] FIG9 is a flowchart of another display method of a virtual reality display device provided in an embodiment of the present application. The method can be applied to the virtual reality display device provided in the above embodiment. The method includes the following steps:

[0227] Step 901: Determine whether the scene type of the display scene of the display unit is a plane scene.

[0228] When the scene type is a plane scene, step 902 is executed. When the scene type is not a plane scene but a three-dimensional scene or a game scene, step 905 is executed.

[0229] Step 902: Determine whether the power level of the power supply component is greater than the target power level.

[0230] When the scene type is a plane scene, the control component can obtain the power in the power component at the current moment, and execute step 903 when the current power obtained is greater than the target power, and execute step 904 when the current power obtained is less than or equal to the target power.

[0231] In addition, when the scene type is a plane scene, the control component can turn off the gaze point sensing unit.

[0232] Step 903: Control the resolution of the display unit to be a first resolution.

[0233] Step 904: Control the resolution of the display unit to be the second resolution.

[0234] The first resolution is a larger resolution than the second resolution. This is because when the power level is greater than the target power level, the power level of the virtual reality device is sufficient, and because the scene type of the current display unit is a flat scene, the control component can set the resolution of the entire display area of ​​the display unit 13 to a larger resolution. For example, the resolution of the entire display area of ​​the display unit 13 can be set to the same resolution as the physical resolution, or to a resolution level that is one level lower than the highest resolution level. For example, the first resolution can be 2664×2880, and the second resolution can be 2368×2560.

[0235] Step 905: Determine whether the head movement speed is greater than the target head movement speed.

[0236] When the head movement speed is greater than the target head movement speed, the control component may execute step 906. When the head movement speed is less than or equal to the target head movement speed, the control component may execute step 907.

[0237] Step 906: Turn off the gaze point sensing unit.

[0238] When the head movement speed is greater than the target head movement speed, the control component can turn off the gaze point sensing unit.

[0239] Step 907: Turn on the gaze point sensing unit. Execute step 908.

[0240] When the head movement speed is not greater than the target head movement speed, the control component can turn on the gaze point sensing unit.

[0241] Step 908: Determine whether the eye movement velocity is greater than the target eye movement velocity.

[0242] When the currently acquired eye movement velocity is greater than the target eye movement velocity, the control component may execute steps 909 and 911. When the currently acquired eye movement velocity is less than or equal to the target eye movement velocity, the control component may execute step 910.

[0243] Step 909: Control the gaze point sensing unit to detect the user's eye movement velocity at a first shutter speed.

[0244] When the eye movement velocity is greater than the target eye movement velocity, the control component may control the gaze point sensing unit to detect the user's eye movement velocity at a first shutter speed.

[0245] Step 910: Control the gaze point sensing unit to detect the user's eye movement velocity at a second shutter speed.

[0246] When the eye movement velocity is less than or equal to the target eye movement velocity, the control component may control the gaze point sensing unit to detect the user's eye movement velocity at a second shutter speed, where the second shutter speed is less than the first shutter speed.

[0247] Step 911: Determine the scene complexity.

[0248] The control component can determine the complexity of the scene. For example, the complexity of the scene can be determined based on the number of vertices of the three-dimensional objects in the scene. For details, please refer to other embodiments of the present application, and the embodiments of the present application will not be repeated here.

[0249] Step 912: In response to the scene complexity being in the first interval, setting the ratio of the gaze area to the display area to a first ratio value.

[0250] Step 913: In response to the scene complexity being in the second interval, setting the ratio of the gaze area to the display area to a second ratio value.

[0251] The minimum value of the second interval is greater than the maximum value of the first interval, and the second ratio value is less than the first ratio value.

[0252] Step 914: In response to the scene complexity being in the third interval, setting the ratio of the gaze area to the display area to a third ratio value.

[0253] The minimum value of the third interval is greater than the maximum value of the second interval, and the third ratio value is less than the second ratio value.

[0254] Among them, the first interval, the second interval and the third interval can be determined based on the threshold of scene complexity. For example, the first interval can be 0≤t<0.1q, where q is the threshold of scene complexity, and the scene complexity can be determined by the total number of vertices of three-dimensional objects in the rendered picture; the second interval can be 0.1q≤t<0.5q; and the third interval can be 0.5q≤t≤q.

[0255] In addition, when the ratio of the gaze area to the display area is a first ratio value, the control component can also determine whether the rendering frame rate is less than the target frame rate; the target frame rate can be a preset frame rate, at which the virtual reality device can achieve a smooth display effect, for example, the target frame rate can be 60 Hz to 240 Hz, such as 90 Hz, 120 Hz, etc.

[0256] In response to the rendering frame rate being less than the target frame rate, the size of the gaze area is adjusted so that the proportion of the gaze area to the display area is a second ratio. When the rendering frame rate is less than the target frame rate, indicating that the current system load is too high to ensure a smooth display, the control component can reduce the proportion of the gaze area to the display area from the first ratio to the second ratio to reduce the system load. This approach ensures the rendering frame rate by reducing the size of the gaze area, improving the user's viewing experience and avoiding any perceived lag or stuttering.

[0257] When the proportion of the gaze area to the display area is a second proportion value, determine whether the rendering frame rate is greater than the target frame rate; after reducing the proportion of the gaze area to the display area, the control component can again determine whether the rendering frame rate is greater than the target frame rate to confirm whether the above operation achieves the effect of ensuring the rendering frame rate.

[0258] In response to the rendering frame rate being less than the target frame rate, the size of the gaze area is adjusted so that the proportion of the gaze area to the display area is a third ratio value. When the rendering frame rate still meets the target frame rate, the control component can further reduce the proportion of the gaze area to the display area from the second ratio value to the third ratio value.

[0259] In addition, the proportion of the gaze area to the display area can have a minimum proportion value, which can be the proportion of the area that can be clearly seen when the human eye is gazing at it in the display area. The control component can reduce the proportion of the gaze area to the display area in the above manner until it is reduced to the minimum proportion value.

[0260] In summary, the embodiments of the present application provide a method for controlling a virtual reality display device, by obtaining the eye movement speed, head movement speed, rendering frame rate of the user wearing the bracket and the hardware resource occupancy rate of the virtual reality display device, and adjusting the display resolution of the display unit based on at least one parameter among the eye movement speed, head movement speed, rendering frame rate and hardware resource occupancy rate through a control component, so that the virtual reality display device can display with different resolutions based on different situations, thereby solving the problem of poor flexibility of the virtual reality display device during display in the related art and achieving the effect of improving the flexibility of the virtual reality display device during display.

[0261] In addition, an embodiment of the present application also provides a virtual reality display device, which may include a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the display method of the virtual reality display device as described above.

[0262] In this application, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0263] In this application, the term "at least one of A and B" simply describes an association relationship between associated objects, indicating that three possible relationships exist. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. Similarly, "at least one of A, B, and C" means that seven possible relationships exist, indicating: A exists alone, B exists alone, C exists alone, A and B exist simultaneously, A and C exist simultaneously, C and B exist simultaneously, and A, B, and C exist simultaneously. Similarly, "at least one of A, B, C, and D" means that fifteen possible relationships exist, indicating: A exists alone, B exists alone, C exists alone, D exists alone, A and B exist simultaneously, A and C exist simultaneously, A and D exist simultaneously, C and B exist simultaneously, D and B exist simultaneously, C and D exist simultaneously, C and D exist simultaneously, A, B, and C exist simultaneously, A, B, and D exist simultaneously, A, C, and D exist simultaneously, B, C, and D exist simultaneously, and A, B, C, and D exist simultaneously.

[0264] In this application, the terms "first", "second", "third" and "fourth" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more, unless otherwise expressly limited.

[0265] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the user's head movement velocity and eye movement velocity data involved in this application are all obtained with full authorization.

[0266] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0267] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0268] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0269] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A virtual reality display device, characterized in that, the virtual reality display device includes: a control component, a bracket, a display unit, a first sensing component, and a second sensing component, and the display unit is located on the bracket; the first sensing component is used to obtain eye movement data of a user wearing the bracket; the second sensing component is used to obtain head movement data of a user wearing the bracket; the control component is used to determine the eye movement speed based on the eye movement data and determine the head movement speed based on the head movement data; the control component is used to obtain the rendering frame rate of the display screen of the display unit and the hardware resource occupancy rate of the virtual reality display device; the control component is further used to adjust the display resolution of the display unit based on at least one of the eye movement speed, the head movement speed, the rendering frame rate, and the hardware resource occupancy rate.

2. The virtual reality display device according to claim 1, characterized in that, the virtual reality display device further includes a power supply component; the control component is further used to obtain the power of the power supply component, the scene type of the display scene of the display unit, and the model complexity in the display screen; the control component is used to adjust the display resolution of the display unit based on at least one of the eye movement speed, the head movement speed, the rendering frame rate, the hardware resource occupancy rate, and at least one of the power, the scene type, and the model complexity.

3. The virtual reality display device according to claim 2, characterized in that, the display resolution of the display unit has multiple resolution levels, and the size of the resolution level is positively correlated with the size of the display resolution of the display unit; the control component is used to: determine a comprehensive parameter based on the eye movement speed, the head movement speed, the rendering frame rate, the power, the hardware resource occupancy rate, and the model complexity, and the eye movement speed, the head movement speed, the rendering frame rate, and the power are negatively correlated with the comprehensive parameter, and the model complexity is positively correlated with the comprehensive parameter; in response to the comprehensive parameter being greater than a specified value, reduce the resolution level of the display unit; in response to the comprehensive parameter being less than or equal to the specified value, increase the resolution level of the display unit.

4. The virtual reality display device according to claim 3, characterized in that, the control component is used to: determine the comprehensive parameter through a preset formula, and the preset formula includes: y = h + e + t + x * f + b + c + g; wherein, y is the comprehensive parameter, h is the normalized parameter of the head movement speed, e is the normalized parameter of the eye movement speed, t is the normalized parameter of the model complexity, f is the normalized parameter of the rendering frame rate, x is the weight of the rendering frame rate, b is the normalized parameter of the power, the hardware resource occupancy rate includes c and g, c is the occupancy rate of the processor in the control component, and g is the occupancy rate of the graphics processor in the control component.

5. The virtual reality display device according to claim 2, characterized in that, The first sensing component includes a fixation point sensing unit, and the control component is configured to: In response to the scene type being a planar scene, turn off the fixation point sensing unit; In response to the power being greater than the target power, control the resolution of the display unit to be a first resolution; In response to the power being less than or equal to the target power, control the resolution of the display unit to be a second resolution, where the second resolution is less than the first resolution.

6. The virtual reality display device according to claim 5, wherein, the control component is configured to: In response to the scene type being a stereoscopic scene, determine whether the head movement speed is greater than the target head movement speed; In response to the head movement speed being greater than the target head movement speed, turn off the fixation point sensing unit; In response to the head movement speed not being greater than the target head movement speed, turn on the fixation point sensing unit.

7. The virtual reality display device according to claim 6, wherein, the control component is configured to: Determine the eye movement speed based on the data collected by the fixation point sensing unit; In response to the eye movement speed being less than the target eye movement speed, determine the fixation area of the user in the display area of the display unit based on the fixation point sensing unit; Control the pixel density of the fixation area in the display area to be a first pixel density, and the pixel density of the area other than the fixation area in the display area to be a second pixel density, where the first pixel density is greater than the second pixel density.

8. The virtual reality display device according to claim 7, wherein, the control component is configured to: In response to the eye movement speed being greater than the target eye movement speed, control the fixation point sensing unit to detect the user's eye movement speed at a first shutter speed; In response to the eye movement speed being less than or equal to the target eye movement speed, control the fixation point sensing unit to detect the user's eye movement speed at a second shutter speed, where the second shutter speed is less than the first shutter speed.

9. The virtual reality display device according to claim 7, wherein, the control component is configured to: In response to the scene complexity being in a first interval, make the proportion of the fixation area in the display area be a first proportion value; In response to the scene complexity being in a second interval, make the proportion of the fixation area in the display area be a second proportion value, where the minimum value of the second interval is greater than the maximum value of the first interval, and the second proportion value is less than the first proportion value; In response to the scene complexity being in a third interval, make the proportion of the fixation area in the display area be a third proportion value, where the minimum value of the third interval is greater than the maximum value of the second interval, and the third proportion value is less than the second proportion value.

10. The virtual reality display device according to claim 9, wherein, the control component is configured to: When the proportion of the fixation area in the display area is the first proportion value, determine whether the rendering frame rate is less than the target frame rate; In response to the rendering frame rate being less than the target frame rate, adjust the size of the fixation area so that the proportion of the fixation area in the display area is the second proportional value; When the proportion of the fixation area in the display area is the second proportional value, determine whether the rendering frame rate is less than the target frame rate; In response to the rendering frame rate being less than the target frame rate, adjust the size of the fixation area so that the proportion of the fixation area in the display area is the third proportional value.

11. The virtual reality display device according to claim 2, wherein, the control component is configured to: set a vertex number acquisition program on each three-dimensional object in the display scene; acquire the number of vertices of the three-dimensional objects in the display scene located in the display screen; determine the model complexity based on the number of vertices.

12. The virtual reality display device according to claim 2, wherein, the control component is configured to: acquire the acceleration data and gyroscope data of the bracket in a specified duration through the second sensing component; obtain a quaternion based on the specified duration, the acceleration data, and the gyroscope data; determine the head movement speed based on the quaternion.

13. The virtual reality display device according to claim 2, wherein, the control component is configured to: acquire the name of the display data corresponding to the display screen, and the name includes data of the scene type; determine the scene type based on the name.

14. A control method for a virtual reality display device, wherein, the virtual reality display device includes: a control component, a bracket, a display unit, a first sensing component, and a second sensor, and the display unit is located on the bracket; the method includes: acquire the eye movement data of a user wearing the bracket through the first sensing component; acquire the head movement data of a user wearing the bracket through the second sensing component; determine the eye movement speed based on the eye movement data, and determine the head movement speed based on the head movement data; acquire the rendering frame rate of the display screen of the display unit and the hardware resource occupancy rate of the virtual reality display device; adjust the display resolution of the display unit based on at least one of the eye movement speed, the head movement speed, the rendering frame rate, and the hardware resource occupancy rate.

15. A virtual reality display device, wherein, the virtual reality display device includes a processor and a memory, and at least one instruction, at least one program, a code set, or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the control method of the virtual reality display device according to claim 14.

16. A virtual reality display device, wherein, the virtual reality display device includes: a control component, a bracket, a display unit, a first sensing component, and a second sensing component, and the display unit is located on the bracket; the control component controls the display unit to display; The first sensor acquires eye movement data of a user wearing the bracket and provides the eye movement data to the control component; The second sensing component acquires head movement data of a user wearing the bracket and provides the head movement data to the control component; The control component determines an eye movement speed based on the eye movement data and determines a head movement speed based on the head movement data; The control component acquires a rendering frame rate of a display screen of the display unit and a hardware resource occupancy rate of the virtual reality display device; The control component adjusts a display resolution of the display unit based on at least one parameter among the eye movement speed, the head movement speed, the rendering frame rate, and the hardware resource occupancy rate.

Images