Head-mounted display device

Abstract

The invention discloses a head-mounted display device, and belongs to the technical field of augmented reality. The head-mounted display device comprises a camera module. The camera module comprises a shell, a lens assembly, a sensing assembly and an anti-shake assembly. An accommodating cavity is formed in the shell; the lens assembly is movably arranged in the accommodating cavity; the sensing assembly and the lens assembly are oppositely arranged; the anti-shake assembly drives the lens assembly to move reversely relative to the sensing assembly based on the moving direction of the head-mounted display device.

Description

Technical Field

[0001] This application belongs to the field of extended reality technology, specifically relating to a head-mounted display device. Background Technology

[0002] Head-mounted displays, such as extended reality (XR) devices, use two RGB cameras (sensors) to capture images of the external environment and render them on a screen. Unlike fixed image acquisition devices, head-mounted displays need to be worn on the user's head, and they will shake as the user's body or head moves. In this situation, the preview image from a head-mounted display is prone to motion blur, affecting the user's visual experience. Summary of the Invention

[0003] The purpose of this application is to provide a head-mounted display device that can at least solve the motion blur problem in the preview screen of the head-mounted display device.

[0004] In a first aspect, embodiments of this application provide a head-mounted display device, including a camera module, the camera module comprising:

[0005] A housing, wherein a receiving cavity is provided within the housing;

[0006] The lens assembly is movably disposed within the receiving cavity;

[0007] A sensing component is disposed opposite to the lens assembly;

[0008] The image stabilization component drives the lens assembly to move in the opposite direction to the sensing assembly based on the direction of movement of the head-mounted display device.

[0009] In this embodiment, since a stabilization component is provided in the camera module of the head-mounted display device, the stabilization component can drive the lens component to move in the opposite direction to the sensing component based on the movement direction of the head-mounted display device. Therefore, the stabilization component can perform motion compensation on the preview image of the head-mounted display device, thereby suppressing motion blur caused by shaking. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the structure of a head-mounted display device according to an embodiment of this application;

[0011] Figure 2 This is a schematic diagram of the movement direction when the image stabilization component drives the lens component to move in the opposite direction to the sensing component according to an embodiment of this application.

[0012] Figure 3This is a schematic diagram of motion compensation based on an image stabilization component in an XR device according to an embodiment of this application;

[0013] Figure 4 This is a schematic flowchart of a control method for a head-mounted display device according to an embodiment of this application;

[0014] Figure 5 This is a schematic diagram of the control compensation curve of the image stabilization component according to an embodiment of this application;

[0015] Figure 6 This is a schematic diagram of the compensation sensitivity curve of the image stabilization component according to an embodiment of this application;

[0016] Figure 7 This is a schematic flowchart illustrating motion compensation of the preview screen of a head-mounted display device using an image stabilization component according to an embodiment of this application;

[0017] Figure 8 This is a schematic diagram of the exposure curve according to an embodiment of this application;

[0018] Figure 9(a) is a schematic preview of a normal long exposure according to an embodiment of this application;

[0019] Figure 9(b) is a schematic preview of a case where the exposure time is shortened according to an embodiment of this application;

[0020] Figure 10 This is a schematic diagram of the photosensitivity curve according to an embodiment of this application;

[0021] Figure 11(a) is a schematic diagram of a preview image obtained by shortening the exposure time without performing sensitivity compensation according to an embodiment of this application;

[0022] Figure 11(b) is a schematic diagram of a preview image obtained after shortening the exposure time and performing sensitivity compensation according to an embodiment of this application;

[0023] Figure 12 This is a schematic flowchart illustrating the control of exposure time and the compensation of photosensitivity according to an embodiment of this application;

[0024] Figure 13 This is a schematic flowchart illustrating the control of a head-mounted display device according to an embodiment of this application;

[0025] Figure 14 This is a schematic diagram of the structure of the control device of the head-mounted display device according to an embodiment of this application;

[0026] Figure 15 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0027] Figure 16This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0028] The attached figures are labeled as follows:

[0029] 10-Head-mounted display device, 101-House, 102-Lens assembly, 103-Sensing assembly, 104-Shake stabilization assembly, 1041-Driver, 1042-Optical image stabilization sensor, 105-Receiving cavity. Detailed Implementation

[0030] When a user wears a head-mounted display device, the device will shake when the user's body or head moves. This can cause motion blur in the image captured by the camera, especially in low-light conditions. Motion blur can further reduce image clarity and severely impact the user's visual experience.

[0031] This application provides a head-mounted display device with an added image stabilization component in its camera module. When a user wears the head-mounted display device, and the user's body or head moves, the image stabilization component can drive the lens assembly to move in the opposite direction to the sensing assembly based on the movement direction of the head-mounted display device. In this way, motion compensation can be performed on the preview image of the head-mounted display device through the image stabilization component, thereby suppressing motion blur caused by shaking.

[0032] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0033] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0034] The head-mounted display device provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0035] Figure 1This is a schematic diagram of the structure of a head-mounted display device 10 according to an embodiment of this application.

[0036] like Figure 1 As shown, the head-mounted display device 10 includes a camera module, which comprises a housing 101, a lens assembly 102, a sensing assembly 103, and an image stabilization assembly 104. The housing 101 has a receiving cavity 105, within which the lens assembly 102 is movably disposed. The sensing assembly 103 is disposed opposite to the lens assembly 102. The image stabilization assembly 104 can drive the lens assembly 102 to move in the opposite direction relative to the sensing assembly 103 based on the direction of movement of the head-mounted display device 10.

[0037] The image stabilization component 104 may include a drive element 1041 (such as a voice coil motor) and an optical image stabilization sensor 1042 (which may be represented as an OIS device). The image stabilization component 104 drives the lens assembly 102 to move in the opposite direction relative to the sensing assembly 103. This can be achieved by the optical image stabilization sensor 1042 driving the lens assembly 102 to move in the opposite direction relative to the sensing assembly 103 via the drive element 1041. The direction of movement of the lens assembly 102 relative to the sensing assembly 103 may include multiple directions. For example, Figure 2 middle( Figure 2 yes Figure 1 (Top view), the direction in which the lens assembly 102 moves relative to the sensing assembly 103 can be any direction on the plane containing the X and Y axes. In this way, when the user moves while wearing the head-mounted display device, the image stabilization component 104 can flexibly drive the lens assembly 102 to move in the opposite direction relative to the sensing assembly 103 according to the actual direction of movement of the head-mounted display device, thereby effectively compensating for motion and suppressing motion blur caused by shaking.

[0038] Taking a head-mounted display device (XR device) as an example, when a user moves while wearing an XR device, the preview screen is prone to motion blur. When a user sees a blurry image on the XR device, the images output from the left and right RGB cameras are essentially unclear, resulting in poor image quality. Based on the technical solution provided in this application's embodiments, after adding a stabilization component to the XR device, please refer to... Figure 3 When a user is wearing an XR device and moving around, the image stabilization component in the XR device can drive the lens component to move in the opposite direction to the sensor component according to the movement direction of the XR device. This can compensate for motion in the preview image of the XR device and effectively suppress motion blur.

[0039] In some embodiments, the head-mounted display device may further include a drive controller. The drive controller is connected to a gyroscope sensor and / or an accelerometer sensor. When the user moves while wearing the head-mounted display device, the gyroscope sensor and accelerometer sensor can collect gyroscope data and acceleration data from the head-mounted display device. The drive controller can acquire the gyroscope data and / or acceleration data and analyze the user's motion state based on this data. When performing motion compensation, the image stabilization component can drive the lens assembly to move in the opposite direction relative to the sensing components based on the user's motion state to adjust the compensation angle for the user's motion. The drive controller may include an Inertial Measurement Unit (IMU) sensor, which is connected to the gyroscope sensor and accelerometer sensor and can acquire the sensing data collected by the gyroscope sensor and accelerometer sensor in real time.

[0040] Image stabilization components typically have a maximum compensation angle when performing motion compensation. In practical applications, considering that the preview screen of a head-mounted display device is a closed environment lacking surrounding environmental reference coordinates, excessive compensation can lead to overly stable images or image lag during user movement while wearing the head-mounted display, causing the user's perception of motion to be inconsistent with the image's movement, thus resulting in dizziness. Therefore, this application embodiment limits the compensation angle of the image stabilization component; that is, the maximum value of the compensation angle during motion compensation must be less than the maximum compensation angle of the image stabilization component.

[0041] In some embodiments, the maximum compensation angle of the image stabilization component is 1.5 degrees. When limiting the compensation angle of the image stabilization component, the maximum value of the compensation angle can be limited to between 0.6 degrees and 0.8 degrees. Optionally, in some more specific embodiments, the maximum compensation angle of the image stabilization component can be limited to 0.7 degrees.

[0042] When image stabilization (ESD) performs motion compensation, generally, the larger the compensation angle, the greater the deviation of the module lens from the center. This results in a slower response time for the ESD component to pull the lens back to the module center, leading to a stronger sense of lag in the image. To avoid this lag affecting the user's visual experience, the drive current of the ESD component (used to drive the ESD component or its driving elements or OIS devices) can be dynamically adjusted based on the ESD component's compensation angle.

[0043] In some implementations, when motion compensation is performed based on the image stabilization component, it can be determined whether the compensation angle of the image stabilization component is less than or equal to a first threshold. If the compensation angle of the image stabilization component is less than or equal to the first threshold, the image stabilization component can be driven with a first drive current. If the compensation angle of the image stabilization component is greater than the first threshold, the first drive current can be increased. In this way, when the compensation angle of the image stabilization component is large, increasing the drive current can improve the compensation sensitivity of the image stabilization component, thereby pulling the lens angle back to the center position more quickly, achieving the optimal threshold for suppressing motion blur and image lag, and reducing the impact on the user's visual experience.

[0044] The first threshold needs to be less than the maximum value of the compensation angle of the image stabilization component. The specific value can be set according to actual needs and is not limited here. In some implementations, when the maximum compensation angle of the image stabilization component is limited to between 0.6 and 0.8 degrees, the first threshold can be between 0.4 and 0.6 degrees. For example, the first threshold could be 0.5 degrees.

[0045] In some implementations, when performing motion compensation based on the image stabilization component, different compensation angles can be adopted according to the motion state of the user wearing the head-mounted display. For example, taking the motion state of the user wearing the head-mounted display as including a stationary state (which can be tripod-mounted or handheld; tripod-mounted state is when the user is not wearing the head-mounted display, and handheld state is when the user is wearing the head-mounted display but is stationary or making slight movements), a walking state, a running state, and a vigorous exercise state with an intensity greater than running as an example, then when performing motion compensation:

[0046] When the user's motion state changes from a stationary state to a walking state, the compensation angle of the anti-shake component is increased based on the first compensation coefficient;

[0047] When the user switches from walking to running, the compensation angle of the image stabilization component is reduced based on the second compensation coefficient.

[0048] When the user switches from running to strenuous exercise, the compensation angle of the image stabilization component is reduced based on the third compensation coefficient.

[0049] Among them, the first compensation coefficient is greater than the second compensation coefficient, and the second compensation coefficient is greater than the third compensation coefficient.

[0050] In other words, when a user switches from a stationary state to walking, the compensation angle can be increased by a larger factor; when switching from walking to running, the compensation angle can be decreased by a smaller factor; and when switching from running to vigorous exercise, the compensation angle can be decreased by an even smaller factor. This way, because the compensation angle of the image stabilization component can be dynamically adjusted according to the user's movement state—different movement states corresponding to different compensation angles—the user's perception of motion can be kept consistent with the compensated image motion, thereby reducing dizziness.

[0051] In practical applications, head-mounted displays are used in low-light environments. In low-light conditions, motion blur in head-mounted displays can be very severe, even causing dizziness and discomfort for users. Therefore, to reduce the visual impact on users, the exposure time of the head-mounted display's sensing components can be adjusted to improve motion blur in low-light environments.

[0052] In some implementations, it may include:

[0053] When the movement amplitude of the head-mounted display device increases, and the movement amplitude of the head-mounted display device is less than or equal to the second threshold, the exposure time of the sensing component is increased.

[0054] When the movement of the head-mounted display device increases and exceeds the second threshold, the exposure time of the sensing component is reduced.

[0055] The movement range of the head-mounted display device can be obtained by analyzing its gyroscope and acceleration data. Since shortening the exposure time can suppress motion blur, while increasing the exposure time can improve image brightness, when the movement range of the head-mounted display device is small (where motion blur is less severe than brightness), i.e., the movement range is less than the second threshold, prioritizing brightness enhancement by increasing the exposure time of the sensing components is preferable. Conversely, when the movement range of the head-mounted display device is large (where motion blur is more severe than brightness), i.e., the movement range is greater than the second threshold, prioritizing motion blur suppression by reducing or shortening the exposure time of the sensing components is preferable. The second threshold can be set according to actual needs and is not specifically limited here.

[0056] In some implementations, when adjusting the exposure time of the sensing component, different exposure times can be used based on the movement state of the user wearing the head-mounted display. Taking the user's movement state as an example, including a stationary state, walking state, running state, and vigorous exercise state with an intensity greater than running, then when adjusting the exposure time:

[0057] When the user's motion state changes from a stationary state to a walking state, the exposure time of the sensing component is increased based on the first adjustment coefficient;

[0058] When the user's movement state changes from walking to running, the exposure time of the sensing component is reduced based on the second adjustment coefficient;

[0059] When the user's exercise state changes from running to strenuous exercise, the exposure time of the sensing component is reduced based on the third adjustment coefficient;

[0060] Among them, the first adjustment coefficient is greater than 0, the second and third adjustment coefficients are both less than 0, and the absolute value of the second adjustment coefficient is less than the absolute value of the third adjustment coefficient.

[0061] In other words, when a user transitions from a stationary state to walking, the exposure time can be increased by a larger factor; when transitioning from walking to running, the exposure time can be decreased by a smaller factor; and when transitioning from running to vigorous exercise, the exposure time can be decreased by a larger factor. This dynamic adjustment of exposure time based on the user's movement state—different movement states correspond to different exposure times, with more vigorous movements requiring shorter exposure times—can effectively reduce motion blur caused by shaking.

[0062] In practical applications, while shortening the exposure time of the sensing components can effectively suppress motion blur in head-mounted displays, it may also darken the image, resulting in a dull preview interface that can cause visual discomfort for users wearing the head-mounted display. Therefore, to avoid affecting the user's visual experience, adaptive compensation can be applied to the photosensitivity of the sensing components.

[0063] In some implementations, it may include:

[0064] The exposure time of the sensing component is adjusted based on the movement range of the head-mounted display device. When the difference between the adjusted exposure time and the preset exposure time of the sensing component is greater than a first preset value, the photosensitivity of the sensing component increases as the movement range of the head-mounted display device increases.

[0065] The exposure time of the sensing component is adjusted based on the movement range of the head-mounted display device. When the difference between the adjusted exposure time and the preset exposure time of the sensing component is less than or equal to a first preset value, the photosensitivity of the sensing component remains unchanged as the movement range of the head-mounted display device increases.

[0066] Because a significant reduction in exposure time has a substantial impact on image brightness, while a small reduction has little or no impact, it's crucial to determine whether the difference between the adjusted exposure time of the sensor and the default exposure time (i.e., the normal exposure time) is greater than or equal to a first preset value before compensating for sensitivity. If so, the sensor's sensitivity can be compensated by increasing it to enhance the brightness of the preview image on the head-mounted display. If not, no compensation is needed, and the sensor's sensitivity remains unchanged.

[0067] The first preset value can be set according to the actual situation, and no specific limitation is made here. Optionally, in some embodiments, the value of the first preset value can be between 1.5ms and 2.5ms. For example, the first preset value can be 2.0ms.

[0068] In some implementations, when adjusting the photosensitivity of the sensing component, different photosensitivity levels can be adopted based on the movement state of the user wearing the head-mounted display. Taking the user's movement state as an example, including a stationary state, walking state, running state, and vigorous exercise state with an intensity greater than running, then when dynamically adjusting the photosensitivity:

[0069] When the user's motion state changes from a stationary state to a walking state, the photosensitivity of the sensing component is increased based on the first photosensitivity compensation coefficient;

[0070] When the user switches from walking to running, the photosensitivity of the sensing component is increased based on the second photosensitivity compensation coefficient.

[0071] When the user switches from running to strenuous exercise, the photosensitivity of the sensing component is increased based on the third photosensitivity compensation coefficient.

[0072] Among them, the first photosensitive compensation coefficient is smaller than the second photosensitive compensation coefficient, and the second photosensitive compensation coefficient is smaller than the third photosensitive compensation coefficient.

[0073] In other words, as a user transitions from a stationary state to walking, the ISO can be increased by a smaller factor; as a user transitions from walking to running, the ISO can be increased by a larger factor; and as a user transitions from running to vigorous exercise, the ISO can be increased by an even larger factor. This allows for dynamic adjustment of ISO based on the user's movement state—different movements correspond to different ISO levels, with more vigorous movement resulting in higher ISO. Therefore, dynamically adjusting ISO can enhance image brightness and prevent the image from appearing dark due to shortened exposure time during periods of significant user movement.

[0074] In this embodiment, since a stabilization component is provided in the camera module of the head-mounted display device, the stabilization component can drive the lens component to move in the opposite direction to the sensing component based on the movement direction of the head-mounted display device. Therefore, the stabilization component can perform motion compensation on the preview image of the head-mounted display device, thereby suppressing motion blur caused by shaking.

[0075] Based on the head-mounted display device provided in this application embodiment, this application embodiment also provides a control method for the head-mounted display device. This method allows for motion compensation of the preview image of the head-mounted display device using an image stabilization component while the user is wearing the device, effectively suppressing motion blur caused by user body or head movements. Specifically, during motion compensation, the maximum compensation angle can be limited to be less than the maximum compensation angle of the image stabilization component to avoid overcompensation or image lag, ensuring that the user does not experience dizziness while wearing the head-mounted display device, thus suppressing motion blur caused by shaking. Furthermore, different compensation angles can be used for motion compensation based on different user movement states, ensuring that the user's own motion perception matches the image movement, further avoiding overcompensation or image lag, and effectively improving the user's visual experience while wearing the head-mounted display device.

[0076] like Figure 4 As shown in the figure, this application provides a control method for a head-mounted display device, which includes the following steps.

[0077] S402: Determine the motion status of the user wearing the head-mounted display device.

[0078] When a user wears a head-mounted display, their movement status can be determined, such as whether they are walking or running.

[0079] In determining a user's motion state, some implementations may include:

[0080] Acquire gyroscope and acceleration data from the head-mounted display device;

[0081] Based on gyroscope data and acceleration data, the motion state of the user wearing the head-mounted display device is determined.

[0082] Gyroscope data and acceleration data can reflect the motion state of the head-mounted display device, and correspondingly, the motion state of the user wearing the device. Optionally, the gyroscope data and acceleration data can be obtained through inertial measurement unit (IMU) sensors in the head-mounted display device.

[0083] After acquiring gyroscope and acceleration data, the motion state of the user wearing the head-mounted display device can be obtained by analyzing and calculating these data (the specific analysis and calculation methods can be found in the relevant technical implementations, which will not be described in detail here).

[0084] In some implementations, the user's motion state includes at least one of the following states:

[0085] The static state (which can be either tripod state or handheld state; tripod state is when the user is not wearing the head-mounted display device, and handheld state is when the user is wearing the head-mounted display device but is stationary or in a state of slight movement);

[0086] Walking state;

[0087] Running state;

[0088] Intense physical activity.

[0089] S404: Based on the user's motion state, the compensation angle of the image stabilization component is controlled to compensate for motion in the preview image of the head-mounted display device. Different motion states correspond to different compensation angles, and the maximum value of the compensation angle is less than the maximum compensation angle of the image stabilization component.

[0090] After determining the user's motion state, the compensation angle of the image stabilization component can be controlled according to the user's motion state to compensate for motion in the preview image of the head-mounted display device and suppress motion blur caused by shaking.

[0091] In related technologies, the stabilization principle of image stabilization components is as follows: based on gyroscope data and acceleration data, the movement direction of the camera sensor CMOS is calculated, and then the image stabilization component pushes the lens offset in the opposite direction to eliminate motion blur, thereby improving image clarity. When using image stabilization components to compensate for motion in the preview image of a head-mounted display, considering that the preview image is a closed environment lacking surrounding environmental reference coordinates, over-compensation can lead to excessive image stabilization or image lag during user movement while wearing the head-mounted display. This can cause the user to perceive a discrepancy between their own motion perception and the image's movement, resulting in dizziness. Therefore, this embodiment limits the compensation angle when using image stabilization components to compensate for motion in the preview image of a head-mounted display. Specifically, the maximum compensation angle must be less than the maximum compensation angle of the image stabilization component. Furthermore, different compensation angles can be used for different motion states. This eliminates motion blur without causing dizziness due to excessive stabilization or image lag.

[0092] In some implementations, the maximum compensation angle of the image stabilization component can be limited to approximately half of the maximum compensation angle of the image stabilization component. For example... Figure 5 As shown, the maximum compensation angle of the image stabilization component is 1.5 degrees. A larger compensation angle results in a more stable image and better control of motion blur. To avoid inconsistencies between the image stabilization component's excessive image stabilization and the user's own motion perception in head-mounted displays, the maximum compensation angle can be controlled at around 0.7 degrees. Furthermore, the compensation angle can be adjusted according to the user's movement state to enhance the consistency between the image motion and the user's perceived motion. For example, as walking speed increases, the compensation angle can be gradually reduced. When entering a running state, the maximum compensation angle can be further limited to around 0.5 degrees, achieving the optimal critical value for suppressing motion blur and dizziness at different stages of movement.

[0093] In some implementations, controlling the compensation angle of the image stabilization component based on the user's motion state may include:

[0094] The compensation coefficient of the image stabilization component is determined based on the user's motion state;

[0095] The compensation angle of the image stabilization component is controlled based on the compensation coefficient.

[0096] The compensation coefficient can be a control factor for the image stabilization components, with different compensation coefficients corresponding to different motion states. Optionally, this compensation coefficient can be... Figure 5 The slope of the OIS control compensation curve is shown. The compensation angle of the image stabilization component is controlled according to the compensation coefficient; this can involve increasing or decreasing the compensation angle, etc.

[0097] In some implementations, the user's motion state may include at least one of a stationary state, a walking state, a running state, and a vigorous exercise state. In this case, the compensation coefficient of the image stabilization component is determined according to the motion state, specifically: when the motion state changes from a stationary state to a walking state, the compensation coefficient is a first compensation coefficient; when the motion state changes from a walking state to a running state, the compensation coefficient is a second compensation coefficient; and when the motion state changes from a running state to a vigorous exercise state, the compensation coefficient is a third compensation coefficient. The first compensation coefficient is greater than the second compensation coefficient, and the second compensation coefficient is greater than the third compensation coefficient. Optionally, the compensation coefficient can be a specific numerical value or a range value, which is not specifically limited here. For example, the first compensation coefficient can be 0.8 to 1, the second compensation coefficient can be -0.8 to -1, and the third compensation coefficient can be -1.5 to -2.

[0098] It should be noted that increasing the compensation coefficient will increase the compensation angle accordingly, and vice versa. In other words, as the user gradually transitions from a static state to a state of vigorous movement, the compensation angle can be appropriately reduced to ensure that the user's motion perception matches the on-screen motion.

[0099] Please see Figure 5 . Figure 5 The image stabilization component's default compensation curve and the control compensation curve actually used in this embodiment are shown. As can be seen from the actual control compensation curve, in the motion range from a stationary state to a walking state, the compensation coefficient can be larger, and correspondingly, the compensation angle can be increased (the maximum compensation angle is limited to within 0.7 degrees), thereby increasing the compensation effect and ensuring that the motion state of the compensated image is consistent with the tripod, handheld, or walking state. In the motion range from a walking state to a running state, because the user accelerates their movement, to ensure that the motion state of the compensated image is consistent with the user's walking or running state, the compensation coefficient can be reduced, and correspondingly, the compensation angle is also reduced. Similarly, in the motion range from a running state to a vigorous movement state, because the user further accelerates their movement, the compensation coefficient can be further reduced, and correspondingly, the compensation angle is also further reduced. Thus, by using different compensation angles in different motion states, the image blurring problem caused by shaking can be effectively suppressed, while ensuring that the motion state of the compensated image is consistent with the user's walking or running state, reducing dizziness.

[0100] In related technologies, when using image stabilization components for motion compensation, the larger the compensation angle, the greater the deviation of the module lens from the center, and the slower the OIS (Optical Image Stabilization) speed at which the lens returns to the center of the module, resulting in a stronger sense of lag in the image. To avoid this lag affecting the user's visual experience, some implementations may further include:

[0101] Determine whether the compensation angle is less than or equal to the first threshold;

[0102] When the compensation angle is less than or equal to the first threshold, the anti-shake component is driven with the first drive current;

[0103] If the compensation angle is greater than the first threshold, the first driving current is increased.

[0104] The first threshold, being less than the maximum compensation angle of the image stabilization component, represents an angle or range within which the compensation sensitivity needs to be increased. Optionally, the first threshold can be between 0.4 and 0.6 degrees. For example, if the maximum compensation angle of the image stabilization component is 1.5 degrees, and the maximum compensation angle of the image stabilization component is limited to 0.7 degrees, then the first threshold could be 0.5 degrees.

[0105] If the compensation angle is less than or equal to the first threshold, it indicates that the image stabilization component has high compensation sensitivity and meets the requirements. In this case, the image stabilization component can be driven with the first driving current, which can be the standard driving current of the image stabilization component. If the compensation angle is greater than the first threshold, it indicates that the compensation sensitivity of the image stabilization component is low. In this case, the driving current to the image stabilization component can be increased to improve the compensation sensitivity of the image stabilization component, thereby pulling the compensated lens angle back to the center position more quickly and achieving the optimal critical value for suppressing motion blur and image lag.

[0106] Figure 6 This is a schematic diagram of the image stabilization component compensation sensitivity curve according to an embodiment of this application. Figure 6 In this design, the maximum compensation angle of the image stabilization component is 1.5 degrees. During motion compensation, the compensation angle is limited to within 0.7 degrees, meaning that regardless of whether the user is stationary or in vigorous motion, the compensation angle will not exceed 0.7 degrees. When performing motion compensation, no increase in drive current to the image stabilization component is needed when the compensation angle is between 0 and 0.5 degrees. When the compensation angle is between 0.5 and 0.7 degrees, the drive current to the image stabilization component can be increased; increasing the drive current will increase the compensation sensitivity of the image stabilization component. Figure 6 As shown, within the 0.5-0.7 degree range, the compensation sensitivity of the image stabilization component increases accordingly after increasing the drive current. Thus, by increasing the drive current to enhance the compensation sensitivity of the drive component, the compensated lens angle can be pulled back to the center position more quickly, achieving the optimal critical value for suppressing motion blur and image lag.

[0107] When increasing the drive current to the image stabilization component, in some embodiments, the increase in drive current can be less than or equal to a first ratio to avoid excessive current increase that could damage the image stabilization component. The first ratio can be determined based on the actual current handling capacity of the image stabilization component, and is not specifically limited here. Optionally, the first ratio can be 10%.

[0108] It should be noted that the image stabilization component in normal mode is controlled at a standard current value. A deviation of ±10% from the standard current value is generally acceptable, but prolonged use in normal mode is not recommended. Therefore, in this embodiment, the drive current to the image stabilization component may be increased only when the compensation angle is greater than a first threshold, and the increase in current is limited to a first ratio.

[0109] To facilitate understanding of how the image stabilization component is used to compensate for motion in the preview image of the head-mounted display device in the embodiments of this application, the following will be used as an example. Figure 7 A more specific embodiment is illustrated below. Figure 7 The illustrated embodiment includes the following steps:

[0110] Step 1: With the user wearing a head-mounted display device, an image motion of 2 degrees was detected due to the user's movement.

[0111] Step 2: Use OIS gimbal devices (anti-shake components) for motion compensation.

[0112] Step 3: Limit the maximum compensation angle of the OIS gimbal device to 0.7 degrees.

[0113] In other words, regardless of whether the user is stationary or in a state of vigorous movement, the compensation angle of the OIS gimbal device will not exceed 0.7 degrees.

[0114] Step 4: Determine which range the current OIS compensation angle falls within in order to determine whether it is necessary to increase the control current (or drive current) of the OIS.

[0115] If the OIS compensation angle is within the range of 0-0.5 degrees, no additional control current is required. If the OIS compensation angle is within the range of 0.5-0.7 degrees, an additional control current is required to improve the OIS sensitivity; the increase in control current should be less than or equal to 10%.

[0116] Step 5: Obtain motion status via IMU.

[0117] Motion states include static, walking, running, and strenuous exercise states. Different OIS compensation coefficients can be used depending on the different motion states.

[0118] For example, the compensation coefficient o_status can be:

[0119] When using a tripod, handheld devices, or while walking: o_status = 0.8~1;

[0120] When transitioning from walking to running: o_status = -0.8 to -1;

[0121] When running at a high level of exertion: o_status = -1.5 to -2.

[0122] Step 6: Control the OIS IC driver to perform motion compensation in the opposite direction of the camera movement to eliminate motion blur in parts of the image caused by motion.

[0123] Based on steps 1 to 6 above, motion of 0 to 0.7 degrees can be eliminated.

[0124] In practical applications, head-mounted displays (HUDs) are used in low-light environments. In low-light conditions, HUDs prioritize brightness during image capture, controlling the maximum exposure time to around 10ms, suppressing noise, and maintaining an ISO sensitivity of around 3000. This control strategy effectively manages brightness, noise, and motion blur when the HUD is stationary. However, in enclosed environments, HUDs lack ambient reference. When a user is wearing the HUD in low-light motion, motion blur in the preview image becomes severe, potentially causing dizziness or discomfort. Therefore, a balance needs to be struck between brightness, noise, and motion blur in low-light motion scenarios to address the motion blur issue in head-mounted displays.

[0125] In view of this, embodiments of this application provide corresponding solutions. Specifically, in some implementations, when using image stabilization components to compensate for motion in the preview image of a head-mounted display device, at least one of the following may be included:

[0126] The exposure time of the sensing components is adjusted according to the user's movement status;

[0127] The photosensitivity of the sensing components is compensated based on the user's movement status.

[0128] In other words, when using image stabilization components to compensate for motion in the preview image of a head-mounted display, the motion blur problem in the preview image can be further improved by adjusting the exposure time and / or sensitivity compensation of the sensor components. Specifically, adjusting the exposure time can involve shortening it, which effectively suppresses motion blur. Compensating for sensitivity can involve increasing the ISO value, which increases image brightness and addresses the issue of insufficient image brightness caused by shortening the exposure time.

[0129] In some implementations, adjusting the exposure time of the sensing component according to the motion state may include:

[0130] The exposure time adjustment factor is determined based on the user's motion state;

[0131] The exposure time of the sensing component is adjusted according to the exposure time adjustment factor.

[0132] Different motion states can correspond to different exposure time adjustment coefficients, and different exposure time adjustment coefficients result in different exposure times. In other words, different exposure times can be set for different motion states to meet the exposure time requirements under different motion conditions, thereby improving motion blur problems under different motion states. The exposure time adjustment coefficient can represent the degree of adjustment of the exposure time, such as whether to increase or decrease the exposure time, and by how much. Based on the exposure time adjustment coefficient, the exposure time of the sensing component is adjusted, for example, by increasing or decreasing the exposure time of the sensing component itself.

[0133] In some implementations, the user's motion state may include at least one of a stationary state, a walking state, a running state, and a vigorous exercise state. In this case, the exposure time adjustment coefficient determined according to the user's motion state may specifically be: a first adjustment coefficient when the motion state changes from a stationary state to a walking state; a second adjustment coefficient when the motion state changes from a walking state to a running state; and a third adjustment coefficient when the motion state changes from a running state to a vigorous exercise state. The first adjustment coefficient is greater than 0, and both the second and third adjustment coefficients are less than 0, with the absolute value of the second adjustment coefficient being less than the absolute value of the third adjustment coefficient. Optionally, the adjustment coefficient can be a specific numerical value or a range value, without specific limitation. For example, the first adjustment coefficient can be 1 to 1.2, the second adjustment coefficient can be -1 to -1.3, and the third adjustment coefficient can be -1.3 to -1.5.

[0134] Once the exposure time adjustment factor is determined, the exposure time of the sensing component can be adjusted according to this factor.

[0135] In some implementations, adjusting the exposure time of the sensing component according to an exposure time adjustment factor may include:

[0136] When the exposure time adjustment factor is greater than 0, the exposure time of the control sensor component is equal to the product of the reciprocal of the exposure time adjustment factor and the default exposure time.

[0137] When the exposure time adjustment coefficient is less than 0, the exposure time of the control sensor component is equal to the sum of the first time and the default exposure time, the first time is equal to the product of the second time and the default exposure time, and the second time is equal to the sum of the exposure time adjustment coefficient and 1.

[0138] As mentioned earlier, the exposure time adjustment factor can be greater than 0 or less than 0 under different motion conditions. Here, the adjustment method for the exposure time can be determined based on whether the exposure time adjustment factor is greater than 0. Specifically, when the exposure time adjustment factor is greater than 0, the exposure time can be determined using the following formula:

[0139] Re-ExpTime=1 / k_status*Timebase.

[0140] When the exposure time adjustment factor is less than 0, the exposure time can be determined using the following formula:

[0141] Re-ExpTime=(k_status+1)*Timebase+Timebase.

[0142] Where Re-ExpTime is the adjusted exposure time, k_status is the exposure time adjustment coefficient, and Timebase is the default exposure time, which can be up to 10ms.

[0143] It should be noted that when running, the exposure time needs to be no less than 5ms to avoid affecting the image and thus the image quality.

[0144] After obtaining the exposure time, the sensing component can be controlled according to the exposure time, that is, the exposure time of the sensing component can be controlled to be the exposure time calculated above.

[0145] Figure 8 This is a schematic diagram of the exposure time curve according to an embodiment of this application. Figure 8 The default exposure time curve and the exposure time curve actually used in the embodiments of this application are shown. It can be seen from the actual exposure time curve that, within the motion range from a stationary state to a walking state, the exposure time adjustment coefficient (which can be considered the slope of the curve) is greater than 0, allowing for a linear increase in exposure time. Within the motion range from a walking state to a running state, because the user accelerates their movement, the exposure time adjustment coefficient needs to be less than 0 to suppress motion blur, thus reducing the exposure time. Similarly, within the motion range from a running state to a vigorous movement state, because the user further accelerates their movement, the exposure time adjustment coefficient can be further reduced, correspondingly further reducing the exposure time. To avoid affecting imaging, the minimum exposure time needs to be greater than or equal to 5ms during running.

[0146] Figure 9(a) shows the preview image obtained under normal long exposure conditions, and Figure 9(b) shows the preview image obtained after shortening the exposure time. Obviously, shortening the exposure time can effectively suppress motion blur in the preview image of the head-mounted display device.

[0147] In the above embodiments, motion blur of the head-mounted display device is effectively suppressed by shortening the exposure time of the sensing component. However, in practical applications, shortening the exposure time may darken the image, and a dim preview interface may cause visual discomfort for users wearing the head-mounted display device. Therefore, adaptive compensation of the photosensitivity of the sensing component is required. Specifically, when compensating for the photosensitivity of the sensing component, compensation can be made according to the user's motion state to match the compensation effect with the user's motion state, thereby improving the user's visual experience.

[0148] In some implementations, compensating for the photosensitivity of the sensing component based on the user's motion state may include:

[0149] Determine whether the difference between the adjusted exposure time and the default exposure time is greater than the first preset value;

[0150] If the difference exceeds a preset threshold, the photosensitivity of the sensing component is compensated based on the user's motion state.

[0151] Considering that in real-world scenarios, a significant reduction in exposure time has a substantial impact on image brightness, while a small reduction has little or no impact, the approach to compensate for sensitivity should first determine if the difference between the adjusted exposure time of the sensor and the default exposure time (i.e., the exposure time under normal conditions) exceeds a first preset value. If so, the sensor's sensitivity is compensated based on the user's motion state; otherwise, no compensation is required. The first preset value can be set according to actual conditions and is not specifically limited here. Optionally, the first preset value can range from 1.5ms to 2.5ms; for example, it could be 2ms.

[0152] When photosensitivity compensation is determined, the photosensitivity of the sensing component is compensated according to the user's motion state. In some implementations, this may include:

[0153] The photosensitivity compensation coefficient is determined based on the motion state;

[0154] The photosensitivity of the sensing component is compensated based on the photosensitivity compensation coefficient.

[0155] Different motion states correspond to different photosensitivity compensation coefficients, and different photosensitivity compensation coefficients result in different photosensitivity values. In other words, different photosensitivity values ​​can be set for different motion states to meet the photosensitivity compensation requirements under different motion conditions, thereby improving motion blur issues. Based on the photosensitivity compensation coefficient, the photosensitivity of the sensing component is compensated, for example, by controlling the increase or decrease of the photosensitivity value.

[0156] In some implementations, the user's motion state may include at least one of a stationary state, a walking state, a running state, and a vigorous exercise state. In this case, the photosensitivity compensation coefficient determined based on the user's motion state may specifically be: a first photosensitivity compensation coefficient when the motion state changes from a stationary state to a walking state; a second photosensitivity compensation coefficient when the motion state changes from a walking state to a running state; and a third photosensitivity compensation coefficient when the motion state changes from a running state to a vigorous exercise state. The first photosensitivity compensation coefficient is less than the second photosensitivity compensation coefficient, and the second photosensitivity compensation coefficient is less than the third photosensitivity compensation coefficient. Optionally, the photosensitivity compensation coefficient can be a specific numerical value or a range value, without specific limitation. For example, the first photosensitivity compensation coefficient can be 1, the second photosensitivity compensation coefficient can be 1.2 to 1.5, and the third photosensitivity compensation coefficient can be 1.5 to 2.

[0157] After obtaining the photosensitivity compensation coefficient, the photosensitivity of the sensing component can be compensated based on the photosensitivity compensation coefficient. In some embodiments, compensating for the photosensitivity of the sensing component based on the photosensitivity compensation coefficient may include:

[0158] The photosensitivity of the control sensor component is equal to the product of the photosensitivity compensation coefficient and the default photosensitivity.

[0159] In other words, the photosensitivity of the sensing component can be determined in the following ways:

[0160] Re-Iso = I_status * Isobase.

[0161] Wherein, Re-Iso is the adjusted photosensitivity value, that is, the photosensitivity of the sensing component needs to be controlled to Re-Iso, I_status is the photosensitivity compensation coefficient, and Isobase is the default photosensitivity value, which can be controlled to a maximum of 3000.

[0162] Figure 10 This is a schematic diagram of the photosensitivity control curve according to an embodiment of this application. Figure 10The diagram illustrates the default ISO curve and the ISO curve actually used in this embodiment. As can be seen from the actual ISO curve, when compensating for ISO, in the motion range from a stationary state to a walking state, the ISO compensation coefficient (which can be considered the slope of the curve) is greater than 0, allowing for a linear increase in ISO. In the motion range from a walking state to a running state, because the user accelerates their movement, the exposure time is shortened to suppress motion blur. Therefore, the ISO compensation coefficient needs to be increased to increase the ISO value, thereby improving image brightness. Similarly, in the motion range from a running state to a vigorous exercise state, because the user further accelerates their movement, the exposure time is further shortened. Therefore, to improve image brightness, the ISO compensation coefficient needs to be further increased, which also further increases the ISO value. Without ISO compensation, the ISO value can be the default ISO.

[0163] Figure 11(a) shows the preview image obtained by shortening the exposure time without performing ISO compensation, and Figure 11(b) shows the preview image obtained by shortening the exposure time and performing ISO compensation. Clearly, by shortening the exposure time and then compensating for ISO, the brightness of the image increases significantly, effectively improving the user's visual experience.

[0164] To facilitate understanding of how the embodiments of this application control exposure time and compensate for photosensitivity, the following will use... Figure 12 A more specific embodiment is illustrated below. Figure 12 The illustrated embodiment includes the following steps:

[0165] Step 1: With the user wearing a head-mounted display device, an image motion of 2 degrees was detected due to the user's movement.

[0166] Step 2: Obtain the default calculated exposure duration.

[0167] According to Figure 8 The default exposure curve shown calculates the default exposure time for the sensor.

[0168] Step 3: Obtain motion status via IMU.

[0169] Movement states include standing still, walking, and running.

[0170] Step 4: Recalculate the exposure time (i.e., exposure duration) based on the motion state.

[0171] Here we can Figure 8 The exposure time is recalculated based on the actual exposure curve used.

[0172] Step 5: Determine the difference between the recalculated exposure time and the default exposure time, and check whether the difference is greater than or equal to 2ms.

[0173] If the time is greater than or equal to 2ms, proceed to step 6; if the time is less than 2ms, proceed to step 7.

[0174] Step 6: Recalculate the sensor ISO sensitivity based on the motion state.

[0175] Step 7: Set the sensor exposure time and ISO sensitivity.

[0176] If ISO compensation is required, the ISO sensitivity here is the sensitivity recalculated in step 6. If ISO compensation is not required, the ISO sensitivity here is the default sensitivity.

[0177] Step 8: Apply the final calculated sensor exposure time and ISO sensitivity value to the sensor hardware to eliminate 0.2 to 0.3 degrees of motion in low-light motion scenes.

[0178] In addressing the motion blur problem in head-mounted display devices, this application combines motion compensation of the image stabilization component with exposure time and sensitivity control strategies. This effectively suppresses motion blur in both moving and low-light motion scenarios, improving the user's visual experience when wearing the head-mounted display. Taking an XR device as the head-mounted display and an optical image stabilization (OIS) gimbal as an example, in some implementations, the specific process can be as follows: Figure 13 As shown.

[0179] Figure 13 This is a schematic flowchart illustrating the control of an XR device according to an embodiment of this application. Figure 13 In the illustrated embodiment, the user moves while wearing the XR device. Assuming the movement produces an image motion of 1.5 degrees, the control flow for the XR device may include:

[0180] Step 1: The OIS gimbal calculates the motion compensation angle value.

[0181] Specifically, first, data from the Gyro gyroscope and accelerometer (ACC) are acquired to determine the motion state. Second, a compensation angle is determined based on the motion state. The maximum value of this compensation angle must be less than the maximum compensation angle of the optical image stabilization (OIS) gimbal. For example, if the maximum compensation angle of the OIS gimbal is 1.5 degrees, the maximum compensation angle can be limited to within 0.7 degrees. Finally, it is determined whether the compensation angle is within a preset range, such as between 0.5 and 0.7 degrees. If it is, the control current (or drive current) of the OIS gimbal is increased; otherwise, no increase in the control current is required. The increase in control current must be less than or equal to a first ratio, for example, less than or equal to 10%, to avoid damaging the components.

[0182] In step 1, after determining the compensation angle and control current, the optical center position of the module can be moved in the opposite direction to perform motion compensation. The maximum compensation for motion is 0.7 degrees.

[0183] Step 2: Controlling exposure time and compensating for ISO sensitivity.

[0184] Specifically, first, the default exposure time and ISO can be determined. Second, based on the motion state determined in step 1, the exposure time is recalculated. Third, it is determined whether the difference between the recalculated exposure time and the default exposure time / duration (or traditional exposure time / duration) is greater than or equal to a preset threshold. If so, the ISO is recalculated; otherwise, recalculation is unnecessary. Finally, based on the recalculated exposure time and (recalculated) ISO, the sensor's exposure time and ISO are controlled.

[0185] Based on step 2, motion of 0.2 to 0.4 degrees can be eliminated.

[0186] Combining steps 1 and 2 can eliminate motion of 0 to 1 degree.

[0187] In addressing the motion blur issue of head-mounted displays, this application employs two methods. First, it utilizes OIS (Optical Image Stabilization) combined with software algorithm control strategies to suppress motion blur caused by shaking while ensuring the user doesn't experience dizziness. Second, it leverages strategies that control sensor exposure and ISO based on changes in motion state to suppress motion blur caused by long exposures in low-light motion scenes while maintaining image brightness. Specifically, optimizing OIS motion angle compensation eliminates motion blur in some motion scenes, resolving the dizziness and image lag issues associated with adding an OIS gimbal to the head-mounted display, while also eliminating some motion blur caused by motion. Optimizing sensor exposure control strategies increases ISO sensitivity while maintaining image brightness, further suppressing motion blur in low-light motion scenes. Therefore, this effectively suppresses motion blur in both motion and low-light motion scenes, improving the user's visual experience when wearing the head-mounted display.

[0188] The control method for a head-mounted display device provided in this application embodiment can be executed by a control device for the head-mounted display device. This application embodiment uses the control device of the head-mounted display device... Device Taking the control method of a head-mounted display device as an example, the control device for the head-mounted display device provided in the embodiments of this application will be described.

[0189] Figure 14 This is a schematic diagram of the structure of the control device 140 of a head-mounted display device according to an embodiment of this application. Figure 14 As shown, in some embodiments, the control device 140 of the head-mounted display device of this application includes a determining module 141 and a control module 142, wherein:

[0190] The determination module 141 is used to determine the motion state of a user wearing a head-mounted display device;

[0191] The control module 142 is used to control the compensation angle of the anti-shake component according to the user's motion state, so as to perform motion compensation on the preview screen of the head-mounted display device.

[0192] Different motion states correspond to different compensation angles, and the maximum value of the compensation angle is less than the maximum compensation angle of the image stabilization component.

[0193] In some embodiments, the determining module 141 is configured to:

[0194] Acquire gyroscope data and acceleration data from the head-mounted display device;

[0195] Based on the gyroscope data and the acceleration data, the motion state of the user wearing the head-mounted display device is determined.

[0196] In some embodiments, the control module 142 is configured to:

[0197] The compensation coefficient of the image stabilization component is determined based on the user's motion state;

[0198] The compensation angle of the image stabilization component is controlled according to the compensation coefficient.

[0199] In some implementations, the user's motion state includes at least one of a stationary state, a walking state, a running state, and a vigorous exercise state;

[0200] Specifically, when the user's movement state changes from the stationary state to the walking state, the compensation coefficient is the first compensation coefficient; when the user's movement state changes from the walking state to the running state, the compensation coefficient is the second compensation coefficient; and when the user's movement state changes from the running state to the vigorous exercise state, the compensation coefficient is the third compensation coefficient. The first compensation coefficient is greater than the second compensation coefficient, and the second compensation coefficient is greater than the third compensation coefficient.

[0201] In some embodiments, the control module 142 is further configured to:

[0202] Determine whether the compensation angle is less than or equal to a first threshold;

[0203] When the compensation angle is less than or equal to the first threshold, the anti-shake component is driven with a first drive current;

[0204] If the compensation angle is greater than the first threshold, the first drive current is increased.

[0205] In some implementations, the first threshold is greater than or equal to the second angle and less than the maximum value of the compensation angle of the image stabilization component, and the increase ratio of the first drive current is less than or equal to the first ratio.

[0206] In some embodiments, the control module 142 is further configured to include at least one of the following:

[0207] The exposure time of the sensing component is adjusted according to the user's motion state;

[0208] The photosensitivity of the sensing component is compensated based on the user's motion state.

[0209] In some embodiments, the control module 142 is configured to:

[0210] The exposure time adjustment factor is determined based on the user's motion state;

[0211] The exposure time of the sensing component is adjusted according to the exposure time adjustment coefficient.

[0212] In some implementations, the user's motion state includes at least one of a stationary state, a walking state, a running state, and a vigorous exercise state;

[0213] Specifically, when the user's movement state changes from the stationary state to the walking state, the exposure time adjustment coefficient is a first adjustment coefficient; when the user's movement state changes from the walking state to the running state, the exposure time adjustment coefficient is a second adjustment coefficient; and when the user's movement state changes from the running state to the vigorous exercise state, the exposure time adjustment coefficient is a third adjustment coefficient. The first adjustment coefficient is greater than 0, and both the second and third adjustment coefficients are less than 0, with the absolute value of the second adjustment coefficient being less than the absolute value of the third adjustment coefficient.

[0214] In some embodiments, the control module 142 is configured to:

[0215] When the exposure time adjustment coefficient is greater than 0, the exposure time of the sensing component is controlled to be equal to the product of the reciprocal of the exposure time adjustment coefficient and the default exposure time.

[0216] When the exposure time adjustment coefficient is less than 0, the exposure time of the sensing component is controlled to be equal to the sum of the first time and the default exposure time, the first time is equal to the product of the second time and the default exposure time, and the second time is equal to the sum of the exposure time adjustment coefficient and 1.

[0217] In some embodiments, the control module 142 is configured to:

[0218] Determine whether the difference between the adjusted exposure time and the default exposure time is greater than or equal to the first preset value;

[0219] If the difference is greater than or equal to the first preset value, the photosensitivity of the sensing component is compensated according to the user's motion state.

[0220] In some embodiments, the control module 142 is configured to:

[0221] The photosensitivity compensation coefficient is determined based on the user's motion state;

[0222] The photosensitivity of the sensing component is compensated according to the photosensitivity compensation coefficient.

[0223] In some implementations, the motion state includes at least one of a stationary state, a walking state, a running state, and a vigorous exercise state;

[0224] Specifically, when the user's movement state changes from the static state to the walking state, the photosensitivity compensation coefficient is a first photosensitivity compensation coefficient; when the user's movement state changes from the walking state to the running state, the photosensitivity compensation coefficient is a second photosensitivity compensation coefficient; and when the user's movement state changes from the running state to the vigorous exercise state, the photosensitivity compensation coefficient is a third photosensitivity compensation coefficient. The first photosensitivity compensation coefficient is smaller than the second photosensitivity compensation coefficient, and the second photosensitivity compensation coefficient is smaller than the third photosensitivity compensation coefficient.

[0225] In some embodiments, the control module 142 is configured to:

[0226] The photosensitivity of the sensing component is controlled to be equal to the product of the photosensitivity compensation coefficient and the default photosensitivity.

[0227] In this embodiment, because an image stabilization component is used to compensate for motion in the preview image of the head-mounted display, and the maximum value of the compensation angle is limited to less than the maximum compensation angle of the image stabilization component, overcompensation or image lag can be avoided. This ensures that the user does not experience dizziness while wearing the head-mounted display, suppressing motion blur caused by shaking. Furthermore, since different compensation angles can be used for motion compensation in different motion states, the user's own motion perception can be consistent with the image motion, further avoiding overcompensation or image lag, thereby effectively improving the user's visual experience while wearing the head-mounted display.

[0228] The control device for the head-mounted display device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0229] The control device for the head-mounted display device in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit the specific operating system used.

[0230] The control device for the head-mounted display device provided in this application embodiment can achieve... Figure 4 , Figure 7 , Figure 12 and Figure 13 To avoid repetition, the various processes implemented in the method implementation examples will not be described again here.

[0231] Optionally, such as Figure 15 As shown, this application embodiment also provides an electronic device 1500, including a processor 1501 and a memory 1502. The memory 1502 stores a program or instructions that can run on the processor 1501. When the program or instructions are executed by the processor 1501, they implement the various steps of the control method embodiment of the head-mounted display device described above and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0232] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0233] Figure 16 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0234] The electronic device 1600 includes, but is not limited to, components such as: radio frequency unit 1601, network module 1602, audio output unit 1603, input unit 1604, sensor 1605, display unit 1606, user input unit 1607, interface unit 1608, memory 1609, and processor 1610.

[0235] Those skilled in the art will understand that the electronic device 1600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 16 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0236] The processor 1610 is used to determine the motion state of a user wearing a head-mounted display device; and to control the compensation angle of the image stabilization component according to the user's motion state, so as to perform motion compensation on the preview screen of the head-mounted display device; wherein different motion states correspond to different compensation angles, and the maximum value of the compensation angle is less than the maximum compensation angle of the image stabilization component.

[0237] By employing image stabilization components to compensate for motion in the preview image of the head-mounted display, and limiting the maximum compensation angle to less than the maximum compensation angle of the stabilization component, overcompensation or image lag can be avoided. This ensures that the user does not experience dizziness while wearing the head-mounted display, suppressing motion blur caused by shaking. Furthermore, because different compensation angles can be applied for different motion states, the user's own motion perception is aligned with the image motion, further preventing overcompensation or image lag, thus effectively improving the user's visual experience when wearing the head-mounted display.

[0238] It is understood that the implementation process of each implementation method mentioned in this embodiment can be referred to Figure 1 The descriptions of the method embodiments shown herein, which can achieve the same or corresponding technical effects, will not be repeated here to avoid repetition.

[0239] It should be understood that, in this embodiment, the input unit 1604 may include a graphics processing unit (GPU) 16041 and a microphone 16042. The GPU 16041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1606 may include a display panel 16061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1607 includes at least one of a touch panel 16071 and other input devices 16072. The touch panel 16071 is also called a touch screen. The touch panel 16071 may include a touch detection device and a touch controller. Other input devices 16072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0240] The memory 1609 can be used to store software programs and various data. The memory 1609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1609 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0241] Processor 1610 may include one or more processing units; optionally, processor 1610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1610.

[0242] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the control method embodiment of the head-mounted display device described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0243] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0244] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the control method embodiment of the head-mounted display device described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0245] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0246] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the control method embodiment for the head-mounted display device described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0247] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0248] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0249] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A head-mounted display device, characterized in that, Includes a camera module, the camera module comprising: A housing, wherein a receiving cavity is provided within the housing; The lens assembly is movably disposed within the receiving cavity; A sensing component is disposed opposite to the lens assembly; The image stabilization component drives the lens assembly to move in the opposite direction to the sensing assembly based on the direction of movement of the head-mounted display device.

2. The head-mounted display device according to claim 1, characterized in that, The head-mounted display device further includes: a drive controller connected to a gyroscope sensor and / or an accelerometer sensor to determine the user's motion state; and an image stabilization component that drives the lens assembly to move relative to the sensing component based on the user's motion state to adjust the compensation angle for the user's motion.

3. The head-mounted display device according to claim 1, characterized in that, The maximum compensation angle of the image stabilization component is between 0.6 degrees and 0.8 degrees.

4. The head-mounted display device according to claim 3, characterized in that, The maximum compensation angle of the image stabilization component is 0.7 degrees.

5. The head-mounted display device according to claim 1, characterized in that, When the compensation angle of the image stabilization component is less than or equal to a first threshold, the image stabilization component is driven with a first drive current. When the compensation angle of the image stabilization component is greater than the first threshold, the first drive current is increased.

6. The head-mounted display device according to claim 5, characterized in that, The first threshold value ranges from 0.4 degrees to 0.6 degrees.

7. The head-mounted display device according to claim 1, characterized in that, When the movement amplitude of the head-mounted display device increases, and the movement amplitude of the head-mounted display device is less than or equal to the second threshold, the exposure time of the sensing component is increased. When the movement amplitude of the head-mounted display device increases and the movement amplitude of the head-mounted display device exceeds the second threshold, the exposure time of the sensing component is reduced.

8. The head-mounted display device according to claim 7, characterized in that, The exposure time of the sensing component is adjusted based on the movement range of the head-mounted display device. When the difference between the adjusted exposure time and the preset exposure time of the sensing component is greater than a first preset value, the photosensitivity of the sensing component increases as the movement range of the head-mounted display device increases. The exposure time of the sensing component is adjusted based on the movement range of the head-mounted display device. When the difference between the adjusted exposure time and the preset exposure time of the sensing component is less than or equal to a first preset value, the photosensitivity of the sensing component remains unchanged as the movement range of the head-mounted display device increases.

9. The head-mounted display device according to claim 8, characterized in that, The first preset value ranges from 1.5ms to 2.5ms.

10. The head-mounted display device according to claim 1, characterized in that, The user's movement state while wearing the head-mounted display includes a stationary state, a walking state, a running state, and a vigorous exercise state with an intensity greater than that of running: When the user's motion state changes from the stationary state to the walking state, the compensation angle of the anti-shake component is increased based on the first compensation coefficient; When the user's movement state switches from walking to running, the compensation angle of the anti-shake component is reduced based on the second compensation coefficient. When the user's exercise state changes from running to vigorous exercise, the compensation angle of the anti-shake component is reduced based on the third compensation coefficient. Wherein, the first compensation coefficient is greater than the second compensation coefficient, and the second compensation coefficient is greater than the third compensation coefficient.

11. The head-mounted display device according to claim 1, characterized in that, The user's movement state while wearing the head-mounted display includes a stationary state, a walking state, a running state, and a vigorous exercise state with an intensity greater than that of running: When the user's motion state changes from the stationary state to the walking state, the exposure time of the sensing component is increased based on a first adjustment coefficient. When the user's movement state changes from walking to running, the exposure time of the sensing component is reduced based on a second adjustment coefficient. When the user's exercise state changes from running to vigorous exercise, the exposure time of the sensing component is reduced based on a third adjustment coefficient. Wherein, the first adjustment coefficient is greater than 0, the second adjustment coefficient and the third adjustment coefficient are both less than 0, and the absolute value of the second adjustment coefficient is less than the absolute value of the third adjustment coefficient.

12. The head-mounted display device according to claim 1, characterized in that, The user's movement state while wearing the head-mounted display includes a stationary state, a walking state, a running state, and a vigorous exercise state with an intensity greater than that of running: When the user's movement state changes from the stationary state to the walking state, the photosensitivity of the sensing component is increased based on the first photosensitivity compensation coefficient. When the user's movement state changes from walking to running, the photosensitivity of the sensing component is increased based on the second photosensitivity compensation coefficient. When the user's exercise state changes from running to vigorous exercise, the photosensitivity of the sensing component is increased based on the third photosensitivity compensation coefficient. Wherein, the first photosensitivity compensation coefficient is less than the second photosensitivity compensation coefficient, and the second photosensitivity compensation coefficient is less than the third photosensitivity compensation coefficient.

Images