Image tone adjustment method, head-mounted display device, and storage medium

Abstract

This invention discloses an image tone adjustment method, a head-mounted display device, and a storage medium. The method includes: acquiring an ambient color temperature value and initial image data; determining tone adjustment parameters for the initial image data based on the ambient color temperature value; adjusting the initial image data according to the tone adjustment parameters to obtain target image data; and displaying the target image data on the display interface of the head-mounted display device. This invention aims to improve the accuracy of head-mounted display devices in reproducing ambient tones and enhance the user experience.

Description

Technical Field

[0001] This invention relates to the field of smart device technology, and in particular to image tone adjustment methods, head-mounted display devices, and storage media. Background Technology

[0002] When head-mounted display devices (such as virtual reality devices) need to display environmental images during use, they generally capture and display scene images of the environment through cameras on the device.

[0003] In the process of displaying environmental images, the images captured by the camera are generally processed by an automatic white balance algorithm to make the displayed image match the color tone of the scene as seen by the user's naked eye. However, in this way, the image processing parameters are estimated from the image data captured by the camera, which can easily lead to a large deviation between the color tone of the final displayed image and the actual environment, affecting the user's interactive experience. Summary of the Invention

[0004] The main objective of this invention is to provide an image tone adjustment method, a head-mounted display device, and a storage medium, aiming to improve the accuracy of the head-mounted display device in reproducing the ambient tone and enhance the user's interactive experience.

[0005] To achieve the above objectives, the present invention provides an image tone adjustment method applied to a head-mounted display device, the image tone adjustment method comprising the following steps:

[0006] Acquire ambient color temperature values ​​and initial image data;

[0007] The tone adjustment parameters of the initial image data are determined based on the ambient color temperature value;

[0008] The target image data is obtained by adjusting the initial image data according to the color adjustment parameters.

[0009] The target image data is displayed on the display interface of the head-mounted display device.

[0010] Optionally, the step of determining the tone adjustment parameters of the initial image data based on the ambient color temperature value includes:

[0011] The tone adjustment matrix of the initial image data is determined based on the ambient color temperature value, and the tone adjustment parameters include the tone adjustment matrix.

[0012] Optionally, the step of determining the tone adjustment matrix of the initial image data based on the ambient color temperature value includes:

[0013] Determine the color temperature range within which the ambient color temperature value falls;

[0014] The preset hue adjustment matrix corresponding to the color temperature range is determined as the hue adjustment matrix.

[0015] Optionally, the initial image data includes RGB signals corresponding to multiple pixels on the display interface, and the step of adjusting the initial image data according to the hue adjustment parameters to obtain the target image data includes:

[0016] Each of the RGB signals is adjusted according to the hue adjustment matrix to obtain the hue value of the corresponding pixel, and the target image data includes the hue values ​​corresponding to the plurality of pixels.

[0017] Optionally, the image tone adjustment method further includes:

[0018] Acquire motion state data of the head-mounted display device;

[0019] When the motion state data reaches the preset conditions, the steps of obtaining the ambient color temperature value and initial image data are executed;

[0020] The preset condition indicates that the displacement of the head-mounted display device is greater than a preset threshold.

[0021] Optionally, after the step of acquiring the motion state data of the head-mounted display device, the method further includes:

[0022] When the motion state data reaches the preset conditions, the steps of obtaining the ambient color temperature value and initial image data are performed when the head-mounted display device is in a moving state;

[0023] The step of determining the tone adjustment parameters of the initial image data based on the ambient color temperature value includes:

[0024] Based on the moving speed of the head-mounted display device corresponding to the moving state, the target correspondence between the color temperature value and the hue adjustment parameter is obtained;

[0025] The hue adjustment parameters corresponding to the ambient color temperature value are determined based on the target correspondence.

[0026] Optionally, the image tone adjustment method further includes:

[0027] Obtain the display mode of the head-mounted display device;

[0028] When the display mode is perspective mode, the steps of obtaining the ambient color temperature value and initial image data are performed.

[0029] Optionally, the step of determining the tone adjustment parameters of the initial image data based on the ambient color temperature value includes:

[0030] Obtain the target perspective rate corresponding to the perspective mode;

[0031] The target correspondence between color temperature values ​​and hue adjustment parameters is obtained based on the target transmittance.

[0032] The hue adjustment parameters corresponding to the ambient color temperature value are determined based on the target correspondence.

[0033] Furthermore, to achieve the above objectives, this application also proposes a head-mounted display device, the head-mounted display device comprising:

[0034] A camera is used to capture initial image data;

[0035] Color temperature sensor, used to collect ambient color temperature values;

[0036] A control device, wherein the camera and the color temperature sensor are both connected to the control device, the control device includes: a memory, a processor, and an image tone adjustment program stored in the memory and executable on the processor, wherein when the image tone adjustment program is executed by the processor, it implements the steps of the image tone adjustment method as described above.

[0037] In addition, to achieve the above objectives, this application also proposes a storage medium storing an image tone adjustment program, which, when executed by a processor, implements the steps of the image tone adjustment method as described in any of the preceding claims.

[0038] This invention proposes an image tone adjustment method applied to head-mounted display devices. This method adjusts the initial image data captured by the camera based on tone adjustment parameters determined by the ambient color temperature value detected by the color temperature sensor on the head-mounted display device to obtain target image data. The target image data is then displayed on the display interface of the head-mounted display device. In this process, the initial image data captured by the camera is no longer adjusted using image processing parameters estimated by an automatic white balance algorithm, but rather using the actual ambient color temperature value detected by the color temperature sensor. This ensures that the target image data displayed by the head-mounted display device can accurately reproduce the scene tone of the device's environment, thereby improving the accuracy of the head-mounted display device in reproducing the ambient tone and enhancing the user experience when viewing the display interface. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the hardware structure involved in the operation of an embodiment of the head-mounted display device of the present invention;

[0040] Figure 2 This is a flowchart illustrating an embodiment of the image tone adjustment method of the present invention;

[0041] Figure 3This is a schematic flowchart of another embodiment of the image tone adjustment method of the present invention;

[0042] Figure 4 This is a flowchart illustrating another embodiment of the image tone adjustment method of the present invention;

[0043] Figure 5 This is a flowchart illustrating another embodiment of the image tone adjustment method of the present invention.

[0044] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0046] This invention provides a head-mounted display device, which can be a display helmet or display glasses, etc. In this embodiment, the head-mounted display device is a virtual reality device. In other embodiments, the head-mounted display device may also be an augmented reality display device.

[0047] In this embodiment of the invention, reference is made to Figure 1 The head-mounted display device includes a control unit 1, a camera 2, and a color temperature sensor 3. Both the camera 2 and the color temperature sensor 3 are connected to the control unit 1. The camera 2 is used to acquire image data of the environment in which the head-mounted display device is located, and the color temperature sensor 3 is used to detect the ambient color temperature value of the environment in which the head-mounted display device is located. The control unit 1 can be used to acquire the data detected by the camera 2 and the color temperature sensor 3.

[0048] In this embodiment, camera 2 is used to acquire color image data. In other embodiments, camera 2 can also be used to acquire black and white image data.

[0049] Furthermore, in one embodiment, the head-mounted display device further includes a motion detection module 4 for detecting motion state data of the head-mounted display device (e.g., velocity, displacement, acceleration, and / or angular velocity). The motion detection module 4 is connected to the control device 1, which can be used to acquire the motion data detected by the motion detection module 4. In this embodiment, the motion detection module 4 is a gyroscope. In other embodiments, the motion detection module 4 may also be other types of detection modules for detecting motion state data, such as a linear velocity sensor.

[0050] In this embodiment of the invention, reference is made to Figure 1The control device 1 includes a processor 1001 (e.g., CPU), a memory 1002, a timer 1003, etc. The components in the control device 1 are connected via a communication bus. The memory 1002 can be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1002 can also be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] like Figure 1 As shown, the memory 1002, which serves as a storage medium, may include an image tone adjustment program. Figure 1 In the apparatus shown, the processor 1001 can be used to call the image tone adjustment program stored in the memory 1002 and execute the relevant steps of the image tone adjustment method in the following embodiments.

[0053] This invention also provides an image tone adjustment method, applied to the aforementioned head-mounted display device.

[0054] Reference Figure 2 This application proposes an embodiment of an image tone adjustment method. In this embodiment, the image tone adjustment method includes:

[0055] Step S10: Obtain the ambient color temperature value and initial image data;

[0056] The specific ambient color temperature value can be obtained by acquiring real-time data from the color temperature sensor on the head-mounted display device, or determined based on multiple data points detected by the color temperature sensor within a target time period.

[0057] The initial image data is the raw image data captured by the camera on the head-mounted display device, which can be obtained by acquiring the data captured by the camera in real time.

[0058] Step S20: Determine the tone adjustment parameters of the initial image data based on the ambient color temperature value;

[0059] The tone adjustment parameters are specifically image processing parameters used to adjust the tone of an image.

[0060] Different ambient color temperatures correspond to different hue adjustment parameters. Specifically, a correspondence between ambient color temperatures and hue adjustment parameters can be established in advance, including mapping relationships, calculation relationships, etc. Based on this correspondence, the hue adjustment parameters corresponding to the current ambient color temperature can be determined.

[0061] In one implementation of this embodiment, different color temperature ranges can be associated with different preset tone adjustment parameters. A target range in which the ambient color temperature value is within more than one color temperature range is determined, and the preset tone adjustment parameters associated with the target range are determined as the tone adjustment parameters of the current initial image data.

[0062] In another implementation of this embodiment, the hue adjustment parameters can also be calculated by substituting the ambient color temperature value into a preset algorithm model.

[0063] Step S30: Adjust the initial image data according to the tone adjustment parameters to obtain the target image data;

[0064] The initial image data includes sub-image data corresponding to multiple pixels on the display interface.

[0065] Adjust the sub-image data corresponding to each pixel in the initial image data according to the tone adjustment parameters to obtain the target sub-image data corresponding to each pixel. The target image data includes multiple sub-image data corresponding to multiple pixels.

[0066] Step S40: Display the target image data on the display interface of the head-mounted display device.

[0067] This invention proposes an image tone adjustment method. This method adjusts the initial image data captured by the camera based on tone adjustment parameters determined by the ambient color temperature value detected by the color temperature sensor on the head-mounted display device to obtain target image data. The target image data is then displayed on the display interface of the head-mounted display device. In this process, the initial image data captured by the camera is no longer adjusted using image processing parameters estimated by an automatic white balance algorithm, but rather by the actual ambient color temperature value detected by the color temperature sensor. This ensures that the target image data displayed by the head-mounted display device can accurately reproduce the scene tone of the environment in which the device is located, thereby improving the accuracy of the head-mounted display device in reproducing the ambient tone and enhancing the user experience when viewing the display interface.

[0068] Furthermore, based on the above embodiments, another embodiment of the image tone adjustment method of this application is proposed.

[0069] In this embodiment, refer to Figure 3 Step S20 includes:

[0070] Step S21: Determine the tone adjustment matrix of the initial image data based on the ambient color temperature value, wherein the tone adjustment parameters include the tone adjustment matrix.

[0071] The tone adjustment matrix is ​​a matrix of parameters used to adjust the color signals in the initial image data. Specifically, the tone adjustment matrix includes adjustment values ​​for one or more color signals corresponding to each pixel in the initial image data.

[0072] Different ambient color temperature values ​​correspond to different hue adjustment matrices.

[0073] In this embodiment, the color temperature range in which the ambient color temperature value lies is determined; the preset tone adjustment matrix corresponding to the color temperature range is determined as the tone adjustment matrix. Specifically, based on the different magnitudes of the color temperature values, at least two preset color temperature ranges are pre-divided. These at least two preset color temperature ranges can be continuous or discontinuous. Each preset color temperature range is associated with a different preset tone adjustment matrix. Based on this, the color temperature range in which the ambient color temperature value lies within the at least two preset color temperature ranges is determined, and the preset tone adjustment matrix associated with this color temperature range is used as the tone adjustment matrix for the current initial image data.

[0074] The target image data can be obtained by adjusting the color signal of the corresponding pixel by adjusting each adjustment value in the hue adjustment matrix.

[0075] In this embodiment, the initial image data is adjusted based on the hue adjustment matrix determined by the ambient color temperature value, which helps to improve the accuracy of the initial image data adjustment, thereby further improving the accuracy of the head-mounted display device in reproducing the ambient hue and enhancing the user interaction experience when the user sees the display interface.

[0076] In other embodiments, the tone adjustment parameter can also be a tone adjustment algorithm. The tone adjustment algorithm is determined based on the ambient color temperature value. Different ambient color temperature values ​​correspond to different tone adjustment algorithms. Based on this, the color signals corresponding to different pixels in the initial image data are processed according to the determined tone adjustment algorithm, and the processed result is used as the target image data.

[0077] Furthermore, in this embodiment, based on step S21, referring to Figure 3 The initial image data includes RGB signals corresponding to multiple pixels on the display interface, and step S30 includes:

[0078] Step S31: Adjust each of the RGB signals according to the hue adjustment matrix to obtain the hue value of the corresponding pixel. The target image data includes the hue values ​​corresponding to the plurality of pixels.

[0079] The tone adjustment matrix includes the color adjustment value for each primary color signal in the RGB signal corresponding to each pixel. By adjusting the corresponding primary color signal in each RGB signal according to the tone adjustment matrix, the tone value for each pixel can be obtained.

[0080] Specifically, the color adjustment value is the adjustment coefficient of the corresponding primary color signal. Multiplying each primary color signal by the corresponding adjustment coefficient in the hue adjustment matrix yields the adjusted target primary color signal. All target primary color signals corresponding to each pixel can determine the hue value corresponding to each pixel.

[0081] In this embodiment, the target image data is obtained by adjusting the RGB signals in the initial image data through a hue adjustment matrix. This ensures that the hue values ​​of the displayed image output by different primary color signals in the target image data can match the actual environmental hue reference, thereby further improving the accuracy of the head-mounted display device in reproducing the environmental hue and enhancing the user interaction experience when the user sees the display interface.

[0082] Furthermore, based on any of the above embodiments, another embodiment of the image tone adjustment method of this application is proposed. In this embodiment, reference is made to... Figure 4 The image tone adjustment method further includes:

[0083] Step S01: Obtain motion state data of the head-mounted display device;

[0084] The motion state data is specifically obtained by acquiring data detected by a motion detection module installed on the head-mounted display device. In this embodiment, the motion detection module is a gyroscope, and the data detected by the gyroscope is used as the motion state data.

[0085] Step S02: When the motion state data reaches a preset condition, the step of acquiring the ambient color temperature value and initial image data is executed; wherein, the preset condition indicates that the displacement of the head-mounted display device is greater than a preset threshold.

[0086] In this embodiment, the motion state data includes angular velocity or acceleration detected by the gyroscope. When the angular velocity or acceleration is greater than or equal to the corresponding preset velocity threshold, it can be determined that the motion state data has met the preset conditions; otherwise, it can be determined that the motion state data has not met the preset conditions.

[0087] When the motion status data does not meet the preset conditions, the image data captured by the camera can be adjusted using the current tone adjustment parameters or preset tone adjustment parameters before being displayed on the display interface.

[0088] In this embodiment, when a significant displacement of the head-mounted display device is detected through motion state data, it indicates that the scene color temperature of the environment in which the head-mounted display device is located may have changed significantly. At this time, the initial image data collected by the camera is adjusted and output according to the method mentioned in the above embodiment. This helps to ensure that the color temperature changes caused by environmental changes during the use of the head-mounted display device are not affected. The head-mounted display device can also accurately reproduce the scene color tone for scene image display, which helps to further improve the realism of the scene image displayed by the head-mounted display device and further improve the user interaction experience.

[0089] Furthermore, in this embodiment, after step S01, the method further includes: when the motion state data reaches a preset condition, and the head-mounted display device is in a moving state, step S10 is executed; step S20 includes: obtaining a target correspondence between the color temperature value and the hue adjustment parameter based on the moving speed of the head-mounted display device corresponding to the moving state; and determining the hue adjustment parameter corresponding to the ambient color temperature value based on the target correspondence.

[0090] Specifically, while the head-mounted display device is in motion, the system acquires at least two color temperature data points—one at the current moment and one prior to the current moment—from a color temperature sensor, and also acquires initial image data captured in real-time by a camera. The ambient color temperature value is then determined based on these two color temperature data points. For example, the average, maximum, or minimum value of the two color temperature data points can be used as the ambient color temperature value.

[0091] Different movement speeds can correspond to different target relationships. Target relationships can include mapping relationships, quantitative relationships, and other forms.

[0092] Specifically, the speed range within which the movement speed occurs can be determined, and the preset correspondence associated with the speed range can be used as the target correspondence. Alternatively, the quantitative relationship between the ambient color temperature value and the hue adjustment parameter in the preset quantitative relationship can be calculated based on the movement speed, and the preset quantitative relationship containing this quantitative relationship value can be used as the target correspondence.

[0093] In this embodiment, the hue adjustment parameters corresponding to the ambient color temperature value are determined by combining the movement speed of the head-mounted display device. This helps to improve the realism of the scene hue displayed in the image when the user is using the head-mounted display device and moving in the scene.

[0094] Furthermore, based on any of the above embodiments, another embodiment of the image tone adjustment method of this application is proposed. In this embodiment, referring to... Figure 5 The image tone adjustment method further includes:

[0095] Step S03: Obtain the display mode of the head-mounted display device;

[0096] The display modes are differentiated based on the display requirements of the head-mounted display device.

[0097] Step S04: When the display mode is perspective mode, perform the step of obtaining ambient color temperature value and initial image data.

[0098] Perspective mode specifically refers to a mode that combines images captured from real-world scenes with virtual objects for display.

[0099] In this embodiment, the head-mounted display device is in perspective mode. The initial image data captured by the camera is displayed after the color tone is adjusted according to the ambient color temperature value detected by the sensor. This helps to improve the realism of the color tone reproduction of the real scene during the perspective display process of the head-mounted display device, thereby further improving the user's experience during the perspective display process.

[0100] Further, step S20 includes: obtaining the target perspective corresponding to the perspective mode; obtaining the target correspondence between the color temperature value and the hue adjustment parameter based on the target perspective; and determining the hue adjustment parameter corresponding to the ambient color temperature value based on the target correspondence.

[0101] Target perspective specifically refers to the transparency of a real-world scene image displayed on a screen. The higher the target perspective, the higher the transparency of the real-world scene image; conversely, the lower the target perspective, the lower the transparency of the real-world scene image.

[0102] Different target perspectives correspond to different target relationships. Specifically, target relationships can include mapping relationships, quantitative relationships, and other forms.

[0103] Specifically, the numerical range of the target transparency can be determined, and the preset correspondence associated with the numerical range can be used as the target correspondence. Alternatively, the quantitative relationship between the ambient color temperature value and the hue adjustment parameter in the preset quantitative relationship can be calculated based on the target transparency, and the preset quantitative relationship containing this quantitative relationship value can be used as the target correspondence.

[0104] In this embodiment, the hue adjustment parameters corresponding to the ambient color temperature value are determined by combining the target perspective rate corresponding to the perspective mode of the head-mounted display device. This helps to improve the realism of the scene hue reproduction of the displayed image when the user displays real-world scenes using different perspective rates.

[0105] Furthermore, in this embodiment, after step S01, the method mentioned above, when the motion state data reaches a preset condition and the head-mounted display device is in a moving state, acquires the ambient color temperature value and the initial image data, can determine the target correspondence based on the moving speed and the target perspective of this embodiment. Different moving speeds and different target perspectives correspond to different target correspondences. Specifically, more than one alternative correspondence can be determined from multiple preset correspondences based on the target perspective, and one of these alternative correspondences is determined as the target correspondence based on the moving speed. Based on this, it is beneficial to improve the realism of the color reproduction of the real scene when the user displays real scene images at different perspectives during the movement of the head-mounted display device, which is beneficial to further improve the user experience of using the head-mounted display device.

[0106] Furthermore, this embodiment of the invention also proposes a storage medium storing an image tone adjustment program, which, when executed by a processor, implements the relevant steps of any of the above embodiments of the image tone adjustment method.

[0107] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0108] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, head-mounted display device, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0110] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An image tone adjustment method, characterized in that, The image tone adjustment method, applied to head-mounted display devices, includes the following steps: Acquire motion state data of the head-mounted display device; When the motion state data reaches the preset condition, when the head-mounted display device is in a moving state and the display mode of the head-mounted display device is perspective mode, the ambient color temperature value and initial image data are obtained. The preset condition indicates that the displacement of the head-mounted display device is greater than a preset threshold. Obtain the target perspective rate corresponding to the perspective mode; Based on the target transmittance, more than one alternative correspondence is determined from multiple preset correspondences, and one of the more than one alternative correspondences is determined as the target correspondence between the color temperature value and the hue adjustment parameter based on the moving speed of the head-mounted display device. Based on the target correspondence, determine the hue adjustment parameters corresponding to the ambient color temperature value; The target image data is obtained by adjusting the initial image data according to the color adjustment parameters. The target image data is displayed on the display interface of the head-mounted display device.

2. The image tone adjustment method as described in claim 1, characterized in that, After the steps of obtaining the ambient color temperature value and initial image data, the method further includes: The tone adjustment matrix of the initial image data is determined based on the ambient color temperature value, and the tone adjustment parameters include the tone adjustment matrix.

3. The image tone adjustment method as described in claim 2, characterized in that, The step of determining the tone adjustment matrix of the initial image data based on the ambient color temperature value includes: Determine the color temperature range within which the ambient color temperature value falls; The preset hue adjustment matrix corresponding to the color temperature range is determined as the hue adjustment matrix.

4. The image tone adjustment method as described in claim 2, characterized in that, The initial image data includes RGB signals corresponding to multiple pixels on the display interface, and the step of adjusting the initial image data according to the hue adjustment parameters to obtain the target image data includes: Each of the RGB signals is adjusted according to the hue adjustment matrix to obtain the hue value of the corresponding pixel, and the target image data includes the hue values ​​corresponding to the plurality of pixels.

5. A head-mounted display device, characterized in that, The head-mounted display device includes: A camera is used to capture initial image data; Color temperature sensor, used to collect ambient color temperature values; A control device, wherein the camera and the color temperature sensor are both connected to the control device, the control device includes: a memory, a processor, and an image tone adjustment program stored in the memory and executable on the processor, wherein when the image tone adjustment program is executed by the processor, it implements the steps of the image tone adjustment method as described in any one of claims 1 to 4.

6. A storage medium, characterized in that, The storage medium stores an image tone adjustment program, which, when executed by a processor, implements the steps of the image tone adjustment method as described in any one of claims 1 to 4.

Images