Chromatic aberration correction method, head-mounted display device and computer readable storage medium

Abstract

The invention discloses a chromatic aberration correction method, head-mounted display equipment and a computer readable storage medium, and the chromatic aberration correction method is applied to the head-mounted display equipment, and comprises the steps: inputting the sampling point information of each pixel in a to-be-displayed image into a pre-constructed target offset model, obtaining first position offset information of a to-be-corrected channel corresponding to each piece of pixel sampling point information; according to the first position offset information of the to-be-corrected channel corresponding to each pixel sampling point information, in combination with an interpolation method, performing interpolation to obtain second position offset information of the to-be-corrected channel corresponding to each pixel point on the to-be-displayed image; according to the second position offset information, an edge color-edge area of the screen is determined, brightness attenuation processing is conducted on the edge color-edge area, and the edge color-edge area refers to a color-edge area formed outside a calibrated target display area corresponding to the screen display frame. According to the method and the device, the transverse chromatic aberration can be corrected in real time with relatively low hardware complexity and operation load.

Description

Technical Field

[0001] The present application relates to the technical field of head-mounted display devices, and in particular to a chromatic aberration correction method, a head-mounted display device, and a computer-readable storage medium. Background Art

[0002] The wavelength range of visible light is about 380 nanometers to 760 nanometers. The refractive index of light of different wavelengths when passing through the lens is also different. When visible light passes through the lens of head-mounted display devices such as VR (Virtual Reality) devices and AR (Augmented Reality) devices, the longer the wavelength of light, the greater the refractive index. For most lenses, blue light has the highest refractive index, followed by green light and red light. Light of different wavelengths is focused at different positions, causing the focal points of blue light, green light and red light to be displaced. This displacement is divided into two types: parallel to the focal plane direction and perpendicular to the focal plane direction. The chromatic aberration caused by the displacement parallel to the focal plane direction is called lateral chromatic aberration, and the chromatic aberration caused by the displacement perpendicular to the focal plane direction is called longitudinal chromatic aberration. Among them, lateral chromatic aberration is the most common and obvious, and with the continuous increase in the resolution of image sensors and the continuous reduction in pixel size, the impact of lateral chromatic aberration is getting bigger and bigger, becoming a problem that needs to be solved urgently for head-mounted display devices.

[0003] At present, there are two types of solutions to eliminate the lateral chromatic aberration of head-mounted display devices: hardware solutions and software solutions. The hardware solution starts with the lens, and eliminates the lateral chromatic aberration through a combination of multiple lenses with different refractive indices. This hardware solution greatly increases the cost of head-mounted display devices. The software method starts with digital image processing, usually performing polynomial fitting on the lateral chromatic aberration, and eliminating the lateral chromatic aberration based on the solved polynomial parameters. This fitting polynomial is usually of high order, and some even reach 11th-order high-order polynomials, resulting in high software calculation complexity and low real-time performance.

[0004] Among the related technologies for eliminating lateral chromatic aberration, hardware solutions have high design costs and complexity, and are not suitable for use in consumer head-mounted display devices. Software solutions have high computational complexity and low real-time performance, making it difficult to perform real-time correction processing on virtual reality images. Summary of the invention

[0005] The main purpose of this application is to provide a chromatic aberration correction method, a head-mounted display device and a computer-readable storage medium, aiming to achieve real-time correction processing of lateral chromatic aberration with lower hardware complexity and operating load.

[0006] To achieve the above object, the present application provides a color difference correction method, which is applied to a head-mounted display device. The method includes:

[0007] Input the information of each pixel sampling point in the image to be displayed into a pre-constructed target offset model, and obtain the first position offset information of the channel to be corrected corresponding to each pixel sampling point;

[0008] According to the first position offset information of the channel to be corrected corresponding to each pixel sampling point, and combining with the interpolation method, interpolate to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed;

[0009] According to the second position offset information, determine the edge color fringe area of the screen, and perform brightness attenuation processing on the edge color fringe area, where the edge color fringe area refers to the color fringe area formed outside the target display area corresponding to the screen display frame.

[0010] In one embodiment, the step of determining the edge color fringe area of the screen according to the second position offset information includes:

[0011] According to the second position offset information, determine the coordinate position of the channel to be corrected corresponding to each pixel point after offset;

[0012] Based on the coordinate position of the channel to be corrected corresponding to each pixel point after offset, determine the edge color fringe area of the screen.

[0013] In one embodiment, the step of performing brightness attenuation processing on the edge color fringe area includes:

[0014] Take the pixel points whose coordinate positions of the channel to be corrected after offset are within the edge color fringe area as target pixel points;

[0015] Perform brightness attenuation processing on the channel to be corrected of the target pixel points, where the brightness attenuation processing is to set the brightness of the channel to be corrected to zero, or reduce the brightness of the channel to be corrected to below a preset brightness value.

[0016] In one embodiment, the method further includes:

[0017] According to the second position offset information, correct the channel to be corrected corresponding to each pixel point. In one embodiment, after the step of correcting the channel to be corrected corresponding to each pixel point according to the second position offset information, the method further includes:

[0018] Through the processing of the electro-optical transfer function EOTF, map the pixel information of each pixel point in the non-linear domain from the non-linear domain to the linear domain, and obtain the pixel information of each pixel point in the linear domain;

[0019] Perform pixel interpolation processing on the pixel information of each pixel point in the linear domain to obtain the pixel information of each pixel point in the linear domain after brightness compensation;

[0020] Through the processing of the opto - electronic transfer function (OETF), map the pixel information of each pixel point in the linear domain after brightness compensation from the linear domain to the non - linear domain, obtain the pixel information of each pixel point in the non - linear domain after brightness compensation and output it.

[0021] In one embodiment, the method further includes:

[0022] In the case of receiving a target brightness compensation instruction, execute the step of mapping the pixel information of each pixel point in the non - linear domain from the non - linear domain to the linear domain through the electro - optical transfer function (EOTF).

[0023] In one embodiment, before inputting the information of each pixel sampling point in the to - be - displayed image into the pre - constructed target offset model, the method further includes:

[0024] Obtain the display module parameter information of the head - mounted display device, where the display module parameter information at least includes light transfer parameters and screen module design parameters;

[0025] According to the display module parameter information, construct the chromatic aberration offset model of the head - mounted display device to obtain the pre - constructed target offset model.

[0026] In one embodiment, the step of constructing the chromatic aberration offset model of the head - mounted display device according to the display module parameter information includes:

[0027] Collect the target dot - matrix virtual image formed after the target dot - matrix light passes through the lens of the head - mounted display device;

[0028] Based on the target dot - matrix virtual image and the display module parameter information, construct the chromatic aberration offset model of the head - mounted display device.

[0029] In one embodiment, the step of constructing the chromatic aberration offset model of the head - mounted display device based on the target dot - matrix virtual image and the display module parameter information includes:

[0030] Obtain the standard dot - matrix image corresponding to the target dot - matrix light, perform image processing on the standard dot - matrix image to obtain the first feature information of each pixel position in the standard dot - matrix image, and perform image processing on the target dot - matrix virtual image to obtain the second feature information of each pixel position in the target dot - matrix virtual image;

[0031] Compare the first feature information with the second feature information to obtain the initial pixel color difference information at each pixel position between the standard dot matrix image and the target dot matrix virtual image;

[0032] Based on the display module parameter information, perform correction processing on the initial pixel color difference information to obtain the target pixel color difference information at each pixel position after correction;

[0033] Based on the target pixel color difference information at each pixel position after correction, construct a color difference offset model of the head-mounted display device.

[0034] In an embodiment, the step of constructing the color difference offset model of the head-mounted display device based on the target pixel color difference information at each pixel position after correction includes:

[0035] Based on the target pixel color difference information at each pixel position after correction, determine the offset amount of the channel to be corrected relative to the reference channel in the target pixel color difference information;

[0036] According to the offset amount of the channel to be corrected relative to the reference channel in the target pixel color difference information, construct a distortion model of the channel to be corrected relative to the reference channel in the target pixel color difference information,

[0037] Based on the distortion model, construct the color difference offset model of the head-mounted display device.

[0038] In addition, to achieve the above object, the present application further provides a color difference correction method, which is applied to a head-mounted display device, and the method includes:

[0039] Based on a pre-constructed color difference correction lookup table, obtain the first position offset information of the channel to be corrected corresponding to each pixel sampling point information in the image to be displayed;

[0040] According to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, and in combination with an interpolation method, interpolate to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed;

[0041] According to the second position offset information, determine the edge color fringe area of the screen, and perform brightness attenuation processing on the edge color fringe area, where the edge color fringe area refers to the color fringe area formed outside the target display area corresponding to the calibration of the screen display frame.

[0042] In addition, to achieve the above object, the present application further provides a head-mounted display device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, it implements the color difference correction method as described above.

[0043] In addition, to achieve the above object, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-described color difference correction method is implemented.

[0044] The embodiments of the present application provide a color difference correction method, a head-mounted display device, and a computer-readable storage medium. The color difference correction method is applied to the head-mounted display device. The technical solution of the embodiments of the present application inputs the information of each pixel sampling point in the image to be displayed into a pre-constructed target offset model, obtains the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, and according to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, combines an interpolation method to interpolate and obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed. Then, according to the second position offset information, the edge color fringe area of the screen is determined, and the brightness attenuation process is performed on the edge color fringe area, where the edge color fringe area refers to the color fringe area formed outside the target display area corresponding to the calibration of the screen display frame, thereby avoiding the hardware solution for eliminating lateral chromatic aberration in the related art, reducing the hardware structure cost and hardware complexity, and also avoiding the software solution based on polynomial fitting, with lightweight operation, reducing the operation load (i.e., reducing the computational complexity), while improving the computational real-time performance. Furthermore, the embodiments of the present application can flexibly, simply, and low-cost correct the color fringe problem in the screen or the occlusion edge area in the application of the head-mounted display device, eliminate the color fringes located at the edge of the field of view, and finally achieve real-time correction processing of the lateral chromatic aberration with a relatively low hardware complexity and operation load. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing the embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0046] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0047] Figure 1 Schematic flow chart provided for the first embodiment of the color difference correction method of the present application;

[0048] Figure 2 Schematic flow chart provided for the second embodiment of the color difference correction method of the present application;

[0049] Figure 3 Schematic flow diagram provided for the third embodiment of the color difference correction method of the present application;

[0050] Figure 4 Schematic flow diagram provided for the fourth embodiment of the color difference correction method of the present application;

[0051] Figure 5 Schematic diagram of the main composition structure of the head-mounted display device in a specific embodiment of the present application;

[0052] Figure 6 Schematic diagram of the overall framework of the color difference correction method in a specific embodiment of the present application;

[0053] Figure 7 Schematic diagram of the interpolation of the pixel-level offset in a specific embodiment of the present application;

[0054] Figure 8 Schematic flow diagram of the edge color fringe elimination in a specific embodiment of the present application;

[0055] Figure 9 Schematic flow diagram of the pixel generation in a specific embodiment of the present application;

[0056] Figure 10 For testing in a specific embodiment of the present application Figure 1 Original schematic diagram;

[0057] Figure 11 For testing in a specific embodiment of the present application Figure 1 Local color fringe schematic diagram;

[0058] Figure 12 For testing in a specific embodiment of the present application Figure 1 Local color fringe elimination schematic diagram;

[0059] Figure 13 For testing in a specific embodiment of the present application Figure 2 Original schematic diagram;

[0060] Figure 14 For testing in a specific embodiment of the present application Figure 2 Local color fringe schematic diagram;

[0061] Figure 15 For testing in a specific embodiment of the present application Figure 2 Global color fringe elimination schematic diagram;

[0062] Figure 16 For testing in a specific embodiment of the present application Figure 2 Local color fringe elimination schematic diagram;

[0063] Figure 17For a specific embodiment of the present application, the test Figure 3 Partial color edge schematic diagram;

[0064] Figure 18 For a specific embodiment of the present application, the test Figure 3 Global color edge elimination schematic diagram;

[0065] Figure 19 For a specific embodiment of the present application, the test Figure 3 Partial color edge elimination schematic diagram;

[0066] Figure 20 Schematic diagram of the device structure of the hardware operating environment involved in the color difference correction method in the embodiment of the present application.

[0067] The realization of the purpose, functional features and advantages of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific embodiments

[0068] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0069] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0070] In the related art, the method of solving the lateral chromatic aberration of the head-mounted display device through hardware starts from the lens, and eliminates the lateral chromatic aberration through the combination of multiple lenses with different refractive indexes. This method relies on the design and manufacture of a complex compound lens system. Although it can effectively reduce or even eliminate the lateral chromatic aberration in theory, the development and manufacture of such a compound lens system require high-precision design and processing technology, which is costly. Moreover, in order to achieve an ideal correction effect, it is often necessary to use a combination of multiple layers of lenses made of different materials, which seriously increases the mechanical complexity and assembly difficulty of the system, restricting its market application range for mass-produced consumer-grade head-mounted display devices.

[0071] The method of solving the lateral chromatic aberration of a head-mounted display device through software starts from digital image processing and eliminates the lateral chromatic aberration by solving the polynomial fitting of the lateral chromatic aberration. In order to accurately simulate and correct the chromatic aberration, this method usually requires a large number of mathematical operations of high-order polynomial fitting, with a relatively high overall computational complexity, a large computational burden on the head-mounted display device, and it is difficult to meet the real-time requirements. Especially in VR / AR application scenarios with extremely high frame rate requirements, it is very easy to lead to poor user experience. Moreover, this software method for eliminating lateral chromatic aberration with high computational load not only affects the performance of the head-mounted display device, but also significantly increases the energy consumption of the head-mounted display device, shortens the battery life, and may require a more powerful processor, further driving up the cost of the head-mounted display device.

[0072] In summary, among the related technologies for eliminating lateral chromatic aberration, the design cost and complexity of the hardware solution are too high and not suitable for use in consumer head-mounted display devices. The software solution has a relatively high computational complexity and low real-time performance, making it difficult to perform real-time correction processing on virtual reality images.

[0073] In view of this, the main solution of the embodiments of the present application is a chromatic aberration correction method applied to a head-mounted display device, including: inputting the information of each pixel sampling point in the image to be displayed into a pre-constructed target offset model to obtain the first position offset information of the channel to be corrected corresponding to each pixel sampling point information; according to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, and combining with an interpolation method, interpolating to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed; according to the second position offset information, determining the edge color fringe area of the screen and performing brightness attenuation processing on the edge color fringe area, where the edge color fringe area refers to the color fringe area formed outside the target display area corresponding to the calibration of the screen display frame.

[0074] Through the above solution, the embodiments of the present application avoid the hardware solution based on lens combination in the related technology, reduce the hardware structure cost and hardware complexity, and also avoid the software solution based on polynomial fitting, with lightweight operation and reduced operation load (i.e., reduced computational complexity), while improving the computational real-time performance. Thus, the embodiments of the present application can flexibly, simply, and low-cost correct the color fringe problem caused by the screen or the occlusion edge area in the application of the head-mounted display device, eliminate the color fringes located at the edge of the field of view, and further achieve real-time correction processing of the lateral chromatic aberration with relatively low hardware complexity and operation load.

[0075] It should be noted that the execution subject of the embodiments of the present application is a head-mounted display device, which may include, but is not limited to, MR (Mixed Reality) devices (such as MR glasses, MR helmets), AR (Augmented Reality) devices (such as AR glasses, AR helmets), VR (Virtual Reality) devices (such as VR glasses, VR helmets), XR (Extended Reality) devices (such as XR glasses, XR helmets), or any head-mounted display device capable of implementing the above functions. The embodiments of the present application do not make specific limitations thereto. The following takes the head-mounted display device as the execution subject as an example to illustrate the following embodiments of the present application.

[0076] To better understand the technical solution of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0077] The present application proposes a chromatic aberration correction method for the first embodiment.

[0078] Please refer to Figure 1 , Figure 1 which is a schematic flowchart provided for the first embodiment of the chromatic aberration correction method of the present application.

[0079] In this embodiment, the chromatic aberration correction method is applied to a head-mounted display device, and the method includes steps S100 to S300:

[0080] Step S100, input the information of each pixel sampling point in the to-be-displayed image into a pre-constructed target offset model, and obtain the first position offset information of the to-be-corrected channel corresponding to each pixel sampling point information;

[0081] Those skilled in the art know that in a head-mounted display device, after an image is transmitted to a display screen, the display screen will drive a pixel light-emitting unit to emit light with corresponding brightness and wavelength according to a graphic signal. These lights are refracted by a lens and then enter the user's eyes, so that the user can view the image. During the process of lens refraction, since lights with different wavelengths have different refractive indexes when passing through the lens, the focal points of the lights are displaced in the direction parallel to the focal plane, resulting in lateral chromatic aberration, thus causing a color edge problem at the edge of the user's field of view (that is, the screen or the occlusion edge area of the head-mounted display device).

[0082] In this embodiment, the to-be-displayed image refers to an image that needs to be transmitted to a display screen for display. A pixel sampling point refers to a pixel point sampled from the to-be-displayed image. Pixel sampling point information refers to the pixel information of the pixel sampling point, and the pixel information mainly includes coordinate position and RGB (Red-Green-Blue) value.

[0083] It should be noted that the channel to be corrected refers to the color channel that needs to be corrected during the process of eliminating the color fringing problem caused by lateral chromatic aberration, and the reference channel refers to the color channel used as a reference benchmark during the process of eliminating the color fringing problem caused by lateral chromatic aberration.

[0084] It can be understood that since green light is located at the center of the visible spectrum, during the refraction of the lens, green light usually exhibits behavior closer to ideal focusing. That is, in lateral chromatic aberration, the chromatic aberration caused by green light is the smallest, and the coordinate positions of the green channel before and after refraction are almost unchanged. Therefore, green light can be used as a reference point to assist in understanding and quantifying the refraction offset of other color lights. Based on this, preferably, the red and blue color channels are set as the channels to be corrected, and the green channel is set as the reference channel.

[0085] It should also be noted that the target offset model is a pre-constructed model used to predict the position offset of the channel to be corrected corresponding to a pixel point relative to the reference channel during the lens refraction process according to the input pixel information. The position offset information is used to describe the position offset of the channel to be corrected corresponding to a pixel point relative to the reference channel during the lens refraction process. Specifically, the position offset information can be the offset amount of the channel to be corrected corresponding to a pixel point relative to the reference channel in the coordinate position after lens refraction, or it can be the coordinate position of the channel to be corrected corresponding to a pixel point after offset after lens refraction. The first position offset information refers to the position offset information of the pixel sampling point in the image to be displayed.

[0086] In this embodiment, by pre-constructing the target offset model of the head-mounted display device, it is ensured that in actual applications, according to the pixel information of multiple pixel sampling points in the image to be displayed, the position offset of the channel to be corrected corresponding to each pixel sampling point relative to the reference channel after the light emitted by the pixel light-emitting unit on the display screen passes through the lens refraction can be predicted through this target offset model. Thus, according to this part of the position offset, the position offset of the channel to be corrected corresponding to all pixel points in the image to be displayed relative to the reference channel is determined, and further, the color fringing at the edges caused by these position offsets is eliminated, ensuring that when the image to be displayed finally enters the user's eyes, there is no color fringing phenomenon at the edge of the user's field of view.

[0087] Step S200, according to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, combined with the interpolation method, interpolate to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed;

[0088] It should be noted that the second position offset information refers to the position offset information of the pixel point in the image to be displayed obtained by interpolation.

[0089] It should also be noted that, in this embodiment, interpolation methods such as linear interpolation, bilinear interpolation, and cubic spline interpolation can be used to expand the acquisition range of the position offset information (i.e., the position offset of the channel to be corrected relative to the reference channel during the refraction of light through the lens) from a small number of pixel sampling points to all pixel points on the image to be displayed, so as to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed.

[0090] Compared with the traditional software solution based on polynomial fitting, in this embodiment, by first obtaining the first position offset information of the channel to be corrected corresponding to a small number of pixel sampling points, and then using the interpolation method to obtain the second position offset information of the channel to be corrected corresponding to all pixel points, while maintaining high data accuracy, the calculation load of the head-mounted display device is reduced, and the ability to perform real-time chromatic aberration correction is greatly improved.

[0091] Step S300: Determine the edge color fringe area of the screen according to the second position offset information, and perform brightness attenuation processing on the edge color fringe area, where the edge color fringe area refers to the color fringe area formed outside the target display area corresponding to the calibration of the screen display frame.

[0092] The screen display frame refers to the boundary of the actual visible area for displaying images in the head-mounted display device. It is the effective range where users can directly see and perceive the image content. That is, only the images within the screen display frame will be directly seen by users, while the images outside the screen display frame are generally not observed by users. However, in actual applications, the light emitted by the pixel light-emitting units on the display screen needs to pass through the refraction of the lens before entering the user's eyes. During the refraction process, due to different wavelengths of light having different refractive indices, the light emitted by the pixel light-emitting units may be distorted after refraction, resulting in color fringes formed outside the target display area corresponding to the calibration of the screen display frame, and then a color fringe phenomenon appears at the edge of the user's field of view, ultimately affecting the overall visual experience of the user using the head-mounted display device.

[0093] In this embodiment, the area position and area size of the target display area need to be calibrated based on the size of the screen display frame of the head-mounted display device. Exemplarily, the area position and area size calibration method of the target display area may be: based on the size of the screen display frame and the display module parameter information of the head-mounted display device, the area position and area size of the target display area are calibrated. Among them, the display module parameter information refers to a set of various parameters that describe the optical and mechanical characteristics of the head-mounted display device, mainly including light transfer parameters and screen module design parameters. Among them, the light transfer parameters refer to a set of parameters that describe the optical characteristics of the lens system, mainly including the refractive index, focal length, field of view (FOV), optical distortion coefficient, etc. of the lens system. The light transfer parameters determine how light is transmitted through the lens system and finally reaches the user's eyes. The screen module design parameters refer to a set of parameters that describe the physical characteristics and performance indicators of the display screen (i.e., the screen), mainly including the resolution, pixel pitch, brightness distribution, gamma curve, etc. of the display screen.

[0094] It should be noted that after the display image within the screen display frame passes through the lens system, the virtual image formed on the human eye side often forms a color edge at the edge of the image due to the different refractive indices of light of different wavelengths. In order to eliminate the color edge, the present embodiment can, exemplarily, in hardware implementation, set a camera on the human eye side to collect the virtual image, and pre-calibrate the target display area based on the screen display frame. The image formed by the light falling into the target display area is the normal image that the user needs to see, and the image formed by the light falling outside the target display area is the color edge area image that the user does not need to see. At this time, the position distribution of pixel points in the virtual image formed after passing through the lens system (which can be collected by the camera) can be used to identify which part of the image belongs to the edge color edge area, and based on the identified edge color edge area, the actual display parameters of each pixel light-emitting unit in the image display device of the head-mounted display device (the display parameters include the brightness values ​​of the red, green and blue channels respectively) are adjusted to perform brightness attenuation processing on the edge color edge area.

[0095] In this regard, after interpolating the second position offset information of the channel to be corrected corresponding to all pixel points in the image to be displayed, this embodiment first determines the coordinate position of the channel to be corrected corresponding to each pixel point after refraction by the lens. Then, based on the coordinate position of the channel to be corrected corresponding to each pixel point after the offset, the edge color fringing area of ​​the screen is determined. Finally, brightness attenuation processing is performed on the pixel points whose coordinate position after the offset of the channel to be corrected is within the edge color fringing area.

[0096] For example, in a feasible implementation manner, the step of determining the edge color fringing area of ​​the screen according to the second position offset information in step S300 may include steps S310 to S320:

[0097] Step S310: Determine the coordinate positions after offset of the channels to be corrected corresponding to each pixel point according to the second position offset information.

[0098] Step S320: Determine the edge color fringe region of the screen based on the coordinate positions after offset of the channels to be corrected corresponding to each pixel point.

[0099] In this embodiment, the second position offset information refers to the offset amount of the channel to be corrected corresponding to the pixel point in the image to be displayed after lens refraction relative to the reference channel in the coordinate position. Since the coordinate position of the reference channel is almost unchanged during the lens refraction process, therefore, the coordinate positions after offset of the channels to be corrected corresponding to each pixel point in the image to be displayed after lens refraction can be calculated through the pixel information of each pixel point in the image to be displayed before lens refraction and the offset amount of the channel to be corrected relative to the reference channel in the coordinate position after lens refraction. Then, in combination with the pre-calibrated target display area corresponding to the screen display frame, the pixel points whose coordinate positions after offset of the channels to be corrected in the image to be displayed after lens refraction are outside the target display area are used as target pixel points, and the coordinate positions after offset of the channels to be corrected corresponding to all the target pixel points are included in the edge color fringe region, thereby determining the edge color fringe region of the screen.

[0100] In addition to this embodiment, when the second position offset information refers to the coordinate positions after offset of the channels to be corrected corresponding to the pixel points in the image to be displayed after lens refraction, it is also possible to directly combine the pre-calibrated target display area corresponding to the screen display frame, and use the pixel points whose coordinate positions after offset of the channels to be corrected in the image to be displayed after lens refraction are outside the target display area as target pixel points, and include the coordinate positions after offset of the channels to be corrected corresponding to all the target pixel points in the edge color fringe region, thereby determining the edge color fringe region of the screen.

[0101] In a feasible embodiment, the step of performing brightness attenuation processing on the edge color fringe region in step S300 may include steps S330 to S340:

[0102] Step S330: Use the pixel points whose coordinate positions after offset of the channels to be corrected are within the edge color fringe region as target pixel points.

[0103] Step S340: Perform brightness attenuation processing on the channels to be corrected of the target pixel points, where the brightness attenuation processing is to set the brightness of the channel to be corrected to zero, or reduce the brightness of the channel to be corrected to below a preset brightness value.

[0104] In this embodiment, after determining the edge color fringe area, the pixel points whose coordinate positions after the offset of the channel to be corrected after the refraction of the lens are located within the edge color fringe area can be directly used as target pixel points, and then the brightness attenuation processing is performed on the channels to be corrected of the target pixel points. Specifically, the brightness of the channels to be corrected of the target pixel points can be set to 0, or the brightness of the channels to be corrected corresponding to the target pixel points can be reduced to a value below a preset brightness value, so as to ensure that when the image to be displayed enters the user's eyes after passing through the lens refraction, the user cannot obviously observe the color fringe phenomenon caused by the offset of the channels to be corrected corresponding to the target pixel points at the edge of the field of view.

[0105] Based on the above first embodiment, a color difference correction method according to the second embodiment of the present application is proposed.

[0106] In the second embodiment of the present application, the same or similar content as that in the above embodiment can be referred to the above introduction and will not be repeated hereinafter.

[0107] Please refer to Figure 2 , Figure 2 which is a schematic flowchart provided for the second embodiment of the color difference correction method of the present application.

[0108] In this embodiment, the color difference correction method may further include step S400:

[0109] Step S400: Correct the channels to be corrected corresponding to each pixel point according to the second position offset information.

[0110] In this embodiment, after interpolating to obtain the second position offset information of the channels to be corrected corresponding to all pixel points in the image to be displayed, in addition to eliminating the color fringes in the edge color fringe area, the color difference of the channels to be corrected corresponding to each display point in the image to be displayed can also be corrected, so as to ensure that when the image to be displayed is transmitted into the user's eyes, the horizontal color difference of the image part in the target display area calibrated by the screen display frame is as low as possible, thereby bringing a better overall visual effect for the user to use the head-mounted display device.

[0111] Specifically, in a feasible implementation manner, after step S400, steps A10 to A30 may further be included:

[0112] Step A10: Through the processing of the electro-optical transfer function EOTF, map the pixel information of each pixel point in the non-linear domain to the linear domain to obtain the pixel information of each pixel point in the linear domain;

[0113] Step A20: Perform pixel interpolation processing on the pixel information of each pixel point in the linear domain to obtain the pixel information of each pixel point in the linear domain after brightness compensation;

[0114] Step A30, by processing the optoelectronic transfer function OETF, the pixel information of each pixel point in the linear domain after brightness compensation is mapped from the linear domain to the nonlinear domain, and the pixel information of each pixel point in the nonlinear domain after brightness compensation is obtained and output.

[0115] It should be noted that, in this embodiment, pixel information mainly refers to pixel values.

[0116] As those skilled in the art know, EOTF (Electro-Optical Transfer Function) is a conversion function used to map pixel values ​​from nonlinear electrical signals to linear optical signals, while OETF (Optical-Electrical Transfer Function) is a conversion function that maps pixel values ​​from linear optical signals back to nonlinear electrical signals.

[0117] This implementation method first extracts the pixel value of each pixel point in the nonlinear domain from the image to be displayed after chromatic aberration correction, and then maps it to the linear domain through EOTF to obtain the pixel value of each pixel point in the linear domain, so as to ensure that the subsequent brightness compensation is based on the real linear brightness value, rather than the compressed or changed nonlinear value, thereby improving the accuracy of brightness compensation. Then, pixel interpolation processing is performed on the pixel value of each pixel point in the linear domain through interpolation methods such as linear interpolation and bilinear interpolation to eliminate the brightness unevenness problem caused by chromatic aberration correction, and obtain the pixel value of each pixel point in the linear domain after brightness compensation. Finally, it is mapped back to the nonlinear domain through OETF to obtain the pixel information of each pixel point in the nonlinear domain after brightness compensation, so as to ensure that the pixel value finally output is suitable for the display characteristics of the head-mounted display device.

[0118] Through this implementation, when brightness compensation is required, the brightness loss of the channel to be corrected that may be caused by pixel interpolation in the nonlinear domain can be effectively solved, thereby avoiding the color cast phenomenon.

[0119] Furthermore, in a possible implementation manner, the chromatic aberration correction method may further include step B10:

[0120] Step B10, when receiving the target brightness compensation instruction, execute: the step of mapping the pixel information of each pixel point in the nonlinear domain from the nonlinear domain to the linear domain through the processing of the electro-optical transfer function EOTF.

[0121] It should be noted that the target brightness compensation instruction is an instruction for triggering brightness compensation of the image to be displayed, and the instruction can be manually triggered by the user or automatically triggered by the head mounted display device.

[0122] In this embodiment, when a target brightness compensation instruction is received, the head-mounted display device is triggered to perform brightness compensation on the image to be displayed, that is, the process of passing through the electro-optical transfer function (EOTF) is executed, and the pixel information of each pixel point in the non-linear domain is mapped from the non-linear domain to the linear domain.

[0123] Through this embodiment, the processing flow of the image to be displayed can be dynamically adjusted according to actual needs, and brightness compensation can be performed on the image to be displayed after color difference correction when the user needs it, thereby improving the quality of the finally presented image, bringing a better visual experience to the user, and skipping relevant steps when not needed, saving computing resources and processing time, thereby improving the overall processing efficiency.

[0124] Based on the above embodiments, a color difference correction method according to the third embodiment of the present application is proposed.

[0125] In the third embodiment of the present application, the same or similar content as the above embodiments can be referred to the above introduction and will not be elaborated hereinafter.

[0126] Please refer to Figure 3 , Figure 3 which is a schematic flow chart provided for the third embodiment of the color difference correction method of the present application.

[0127] In this embodiment, before the step of inputting the information of each pixel sampling point in the image to be displayed into the pre-constructed target offset model in step S100, the color difference correction method may further include steps S500 to S600:

[0128] Step S500, obtaining the display module parameter information of the head-mounted display device, where the display module parameter information at least includes light transfer parameters and screen module design parameters;

[0129] It should be noted that the display module parameter information refers to a set of various parameters describing the optical and mechanical characteristics of the head-mounted display device, mainly including light transfer parameters and screen module design parameters. Among them, the light transfer parameters refer to a set of parameters describing the optical characteristics of the lens system, mainly including the refractive index, focal length, field of view (FOV), optical distortion coefficient, etc. of the lens system. The light transfer parameters determine how light is transmitted through the lens system and finally reaches the user's eyes. The screen module design parameters refer to a set of parameters describing the physical characteristics and performance indicators of the display screen (i.e., the screen), mainly including the resolution, pixel pitch, brightness distribution, gamma curve, etc. of the display screen.

[0130] Step S600, constructing a color difference offset model of the head-mounted display device according to the display module parameter information to obtain a pre-constructed target offset model.

[0131] It should also be noted that in this embodiment, the chromatic aberration offset model is a model used to describe the chromatic aberration phenomenon that occurs in a specific color channel when passing through a lens system. This chromatic aberration offset model is constructed based on the display module parameter information and can predict the position offset of each pixel point in a specific color channel.

[0132] In this embodiment, a chromatic aberration offset model can be constructed for each color channel. Then, in actual application, according to the color offset model corresponding to the channel to be corrected selected for this chromatic aberration correction, the final target offset model can be combined. Alternatively, a chromatic aberration offset model covering all color channels can be directly constructed as the target offset model, and then the output content of the target offset model can be adjusted according to the channel to be corrected in actual application.

[0133] This embodiment makes the chromatic aberration correction method more accurate and effective through a systematic parameter acquisition and model construction process, significantly improving the image quality and user experience, especially in high-demand application scenarios of head-mounted display devices.

[0134] Specifically, in a feasible embodiment, the step of constructing the chromatic aberration offset model of the head-mounted display device according to the display module parameter information in step S600 may include steps S610 to S620:

[0135] Step S610, collect the target dot matrix virtual image formed after the target dot matrix light passes through the lens of the head-mounted display device;

[0136] It should be noted that the target dot matrix light is a dot matrix array of light points arranged according to a specific array, and the target dot matrix virtual image refers to the virtual image formed after the target dot matrix light passes through the lens of the head-mounted display device.

[0137] Step S620, construct the chromatic aberration offset model of the head-mounted display device based on the target dot matrix virtual image and the display module parameter information.

[0138] In this embodiment, step S620 may include steps S621 to S624:

[0139] Step S621, obtain the standard dot matrix image corresponding to the target dot matrix light, perform image processing on the standard dot matrix image to obtain the first feature information of the positions of each pixel in the standard dot matrix image, and perform image processing on the target dot matrix virtual image to obtain the second feature information of the positions of each pixel in the target dot matrix virtual image;

[0140] It should be noted that in this embodiment, the standard dot matrix image refers to the image formed by the target dot matrix light in an ideal state without any optical system changes, that is, the image formed by the arrangement of the light points in the target dot matrix light.

[0141] It should also be noted that the first feature information refers to the feature information of each pixel position in the standard dot matrix image, mainly including coordinate positions, pixel values, etc. The second feature information refers to the feature information of each pixel position in the virtual image of the target dot matrix.

[0142] Step S622: Compare the first feature information with the second feature information to obtain the initial pixel color difference information between the standard dot matrix image and the virtual image of the target dot matrix at each pixel position;

[0143] It should be noted that the initial pixel color difference information refers to the color difference information between the standard dot matrix image and the virtual image of the target dot matrix at each pixel position, mainly including the color differences of each pixel position in the three color channels of red, blue, and green.

[0144] Step S623: Based on the display module parameter information, perform correction processing on the initial pixel color difference information to obtain the target pixel color difference information at each corrected pixel position;

[0145] It should be noted that the target pixel color difference information refers to the color difference information that can more accurately reflect the actual color difference situation between the standard dot matrix image and the virtual image of the target dot matrix at each pixel position after being corrected by the display module parameter information.

[0146] Step S624: Based on the target pixel color difference information at each corrected pixel position, construct a color difference offset model of the head-mounted display device.

[0147] In this embodiment, after obtaining the target pixel color difference information through correction by the display module parameter information, based on this target pixel color difference information, a color difference offset model of the head-mounted display device is constructed to ensure that the finally constructed color difference offset model can accurately reflect the color difference offset situation of the light passing through the lens system in the head-mounted display device, thereby providing a more accurate data basis for subsequent color difference correction, improving the final color difference correction effect, and effectively eliminating possible edge color fringe areas.

[0148] Specifically, in a feasible embodiment, step S624 may include steps C10 to C30:

[0149] Step C10: Based on the target pixel color difference information at each corrected pixel position, determine the offset amount of the channel to be corrected relative to the reference channel in the target pixel color difference information;

[0150] Step C20: According to the offset amount of the channel to be corrected relative to the reference channel in the target pixel color difference information, construct a distortion model of the channel to be corrected relative to the reference channel in the target pixel color difference information;

[0151] Step C30: Based on the distortion model, construct a chromatic aberration offset model for the head-mounted display device.

[0152] In this embodiment, after obtaining the target pixel chromatic aberration information at the corrected pixel positions, the offset of the channel to be corrected relative to the reference channel before and after refraction by the lens can be determined successively in combination with the light transfer parameters for each pixel position, so as to construct a distortion model of the channel to be corrected relative to the reference channel. Finally, based on this distortion model, a chromatic aberration offset model for this head-mounted display device is constructed to ensure that subsequent chromatic aberration correction processing can effectively eliminate color separation in the image.

[0153] It is worth mentioning that, in addition to the above method, the chromatic aberration offset model can also be directly constructed through the offset of the coordinate positions of the three color channels of each light point in the target dot matrix light before and after passing through the lens. Thus, in practical applications, the pixel points corresponding to the light points in the image to be displayed are selected as pixel sampling points, and the first position offset information is determined based on the target offset model constructed based on this chromatic aberration offset model, thereby simplifying the construction process of the chromatic aberration offset model and the target offset model, as well as the computational amount during actual application. Even the chromatic aberration offset model can be directly solidified in the form of a mapping table, and then a target offset model in the form of a mapping table is constructed to further reduce the storage burden of the head-mounted display device.

[0154] In addition, the present application also proposes a chromatic aberration correction method for the fourth embodiment.

[0155] In the fourth embodiment of the present application, for the same or similar content as in the above embodiments, reference can be made to the above introduction and will not be repeated hereinafter.

[0156] Please refer to Figure 4 , Figure 4 which is a schematic flowchart provided for the fourth embodiment of the chromatic aberration correction method of the present application.

[0157] In this embodiment, the chromatic aberration correction method is applied to a head-mounted display device and may include steps S700 to S900:

[0158] Step S700: Based on a pre-constructed chromatic aberration correction lookup table, obtain the first position offset information of the channel to be corrected corresponding to the information of each pixel sampling point in the image to be displayed.

[0159] It should be noted that the chromatic aberration correction lookup table is a pre-constructed mapping table used to reflect the position offset of the channel to be corrected relative to the reference channel before and after refraction of the light passing through the lens of this head-mounted display device. In this embodiment, a target offset model can be pre-constructed and then solidified into a mapping table to obtain the chromatic aberration correction lookup table.

[0160] Step S800: According to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, and in combination with the interpolation method, interpolate to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed;

[0161] Step S900: Determine the edge color fringe area of the screen according to the second position offset information, and perform brightness attenuation processing on the edge color fringe area, where the edge color fringe area refers to the color fringe area formed outside the target display area corresponding to the calibration of the screen display frame.

[0162] In a feasible implementation manner, the step of determining the edge color fringe area of the screen according to the second position offset information in step S900 includes:

[0163] According to the second position offset information, determine the coordinate positions after offset of the channels to be corrected corresponding to each pixel point;

[0164] Based on the coordinate positions after offset of the channels to be corrected corresponding to each pixel point, determine the edge color fringe area of the screen.

[0165] In a feasible implementation manner, the step of determining the edge color fringe area of the screen according to the second position offset information in step S900 includes:

[0166] According to the second position offset information, determine the coordinate positions after offset of the channels to be corrected corresponding to each pixel point;

[0167] Based on the coordinate positions after offset of the channels to be corrected corresponding to each pixel point, determine the edge color fringe area of the screen.

[0168] In a feasible implementation manner, the color difference correction method of the fourth embodiment further includes:

[0169] Correct the channels to be corrected corresponding to each pixel point according to the second position offset information.

[0170] In a feasible implementation manner, after the step of correcting the channels to be corrected corresponding to each pixel point according to the second position offset information, the color difference correction method further includes:

[0171] Through the processing of the electro-optical transfer function EOTF, map the pixel information of each pixel point in the non-linear domain from the non-linear domain to the linear domain to obtain the pixel information of each pixel point in the linear domain;

[0172] Perform pixel interpolation processing on the pixel information of each pixel point in the linear domain to obtain the pixel information of each pixel point in the linear domain after brightness compensation;

[0173] Through the processing of the opto - electronic transfer function (OETF), the pixel information of each pixel point after brightness compensation in the linear domain is mapped from the linear domain to the non - linear domain, and the pixel information of each pixel point after brightness compensation in the non - linear domain is obtained and output.

[0174] In a feasible implementation manner, the color difference correction method of the fourth embodiment further includes:

[0175] When receiving the target brightness compensation instruction, execute the step of: through the processing of the electro - optical transfer function (EOTF), mapping the pixel information of each pixel point in the non - linear domain from the non - linear domain to the linear domain.

[0176] The color correction method provided in this embodiment can perform real - time correction processing on the lateral color difference with relatively low hardware complexity and operating load. Compared with the prior art, the beneficial effects of the color correction method provided in this embodiment are the same as those of the color difference correction method provided in the above - mentioned embodiment, and other technical features in the color correction method provided in this embodiment are the same as the features disclosed in the method of the above - mentioned embodiment, and will not be elaborated here.

[0177] To facilitate the understanding of the technical concept or technical principle of the color difference correction method of the above - mentioned embodiment of the present application, a specific embodiment is listed:

[0178] As Figure 5 shown, in this specific embodiment, the head - mounted display device includes an application processor, a display driver chip, a display screen, and a lens. Among them, the application processor is responsible for performing distortion correction on the image captured by the camera or the image rendered by the graphics renderer (i.e., the image to be displayed), and transmitting it to the display driver chip. The display driver chip receives the image signal transmitted by the application processor, corrects the color difference of the image signal, and transmits the image signal after color difference correction to the display screen. After receiving the image signal after color difference correction transmitted by the display driver chip, the display screen will drive the pixel light - emitting units thereon to emit light with corresponding brightness and wavelength according to this signal, and these lights are refracted by the lens and then enter the human eye.

[0179] As Figure 6As shown in the figure, in this specific embodiment, the color difference correction method can be mainly divided into four parts: the construction of an offset model (i.e., the target offset model), coordinate mapping, pixel generation, and edge color fringe elimination. In this specific embodiment, an offset model is constructed in advance based on prior information such as the optical design parameters (i.e., light transfer parameters) of the head-mounted display device, the screen design parameters (i.e., screen module design parameters), and the shooting results of the virtual image (i.e., the target dot matrix virtual image). This model stores the coordinate mapping relationship between the red and blue channels (i.e., the channels to be corrected) and the green channel (i.e., the reference channel) according to the sampling point information configured in the lookup table (i.e., pixel sampling point information). In actual applications, after the input image, the coordinate mapping module calculates the offset coordinate positions of the red and blue channels of each pixel point relative to the green channel according to the lookup table, that is, according to the first position offset information of the channels to be corrected corresponding to each pixel sampling point information, and combines the interpolation method to interpolate and obtain the second position offset information of the channels to be corrected corresponding to each pixel point on the image to be displayed. The edge color fringe elimination module calculates the pixel positions related to the edge color fringe area according to the lookup table, that is, determines the edge color fringe area of the screen according to the second position offset information. The pixel generation module receives the coordinate positions of the red and blue channels transmitted by the coordinate mapping module, calculates the pixel values at the new coordinates, and at the same time, it also receives the pixel position information transmitted by the edge color fringe elimination module and controls the light emission intensity of these pixels, that is, corrects the channels to be corrected corresponding to each pixel point, performs brightness attenuation processing on the edge color fringe area, and performs brightness compensation on each pixel point. Finally, an image after color difference correction and brightness compensation is output.

[0180] Specifically, the construction process of the offset model is as follows:

[0181] First of all, a distortion model for the red, green, and blue channels needs to be established. The barrel distortion is expressed by formula (1):

[0182] p′=BarrelTrans(p,"color channel") (1)

[0183] Where p represents the coordinate position of the pixel point, color channel represents the color channel, BarrelTrans() represents the functional expression of the barrel distortion, and p ′ represents the coordinate position of the pixel point after barrel distortion in a certain color channel

[0184] For pincushion distortion, it is expressed by formula (2):

[0185] p=PincushionTrans(p′,"color channel") (2)

[0186] Among them, PincushionTrans() represents the functional expression of pincushion distortion.

[0187] As known to those skilled in the art, for barrel distortion and pincushion distortion of the same color channel, they should be inverse mappings of each other.

[0188] During the construction process of the offset model, the overall operation can be divided into four steps. One optional solution is to use a lookup table to store the offsets of the red and blue channels relative to the green channel. The specific operation process is as follows:

[0189] Step S11: Perform pincushion distortion on the original pixel position p G according to the distortion model of the green channel to obtain the distorted position p'. G .

[0190] Among them, the distortion model of the green channel is expressed by formula (3):

[0191] p' G = PincushionTrans(p G ,"G")(3)

[0192] Step S12: Perform barrel distortion on the result p' of pincushion distortion G respectively according to the distortion models of the red and blue channels to obtain the distorted positions p R and p B .

[0193] Among them, the distortion model of the red channel is expressed by formula (4):

[0194] p R = BarrelTrans(p' G ,"R")(4)

[0195] The distortion model of the blue channel is expressed by formula (5):

[0196] p B = BarrelTrans(p' G ,"B") (5)

[0197] Step S13: Calculate the offsets of the distorted positions p R 、p B of the red and blue channels relative to the original pixel position p G .

[0198] Among them, the offset offset R of the distorted position p G of the red channel relative to p RG can be calculated by formula (6):

[0199] offset RG = p R - p G (6)

[0200] The position p after the blue channel distortion B Relative to p G The offset offset BG Can be calculated by formula (7):

[0201] offset BG = p B - p G (7)

[0202] S14: Sample offset RG And offset BG To obtain the look-up tables LUT RG Between red-green and blue-green BG .

[0203] That is, for the sampling points configured in the look-up table, through the above steps S11 to S13 in sequence, the corresponding offset of each sampling point is obtained RG And offset BG , and finally the look-up table LUT of the position offset of the red channel relative to the green channel before and after refraction is obtained by summarization RG , and the look-up table LUT of the position offset of the blue channel relative to the green channel before and after refraction BG .

[0204] In this specific embodiment, the coordinate mapping module needs to calculate the offsets of the corresponding red and blue channels relative to the green channel for each pixel point before and after refraction. As can be seen from step S14, the look-up table stores the offset accuracy in units of image blocks after downsampling. Therefore, it is necessary to magnify the offset to the pixel level by means of interpolation (that is, the interpolation method). That is, according to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, combined with the interpolation method, the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed is interpolated

[0205] As Figure 7 Shown, in an alternative solution, the pixel-level offset can be interpolated by the bilinear interpolation method. In Figure 7In it, the left side shows the offsets of 9 sampling points in the offset lookup table in units of image blocks, and the right side shows the process of interpolating to obtain the pixel-level offset by the bilinear interpolation method. Specifically, this optional solution interpolates the offset of a pixel point by using the offsets of four block-level sampling points, namely the upper left (A), lower left (B), upper right (C), and lower right (D) of the pixel point (i.e., the red dot in the figure), through the bilinear interpolation method. The specific interpolation process is shown in formula (8):

[0206] E = ((A * (y - dis_y) + B * dis_y) * (x - dis_x) + (C * (y - dis_y) + D * dis_y) * dis_x) / (x * y) (8)

[0207] Among them, E represents Figure 7 the offset of the red dot in it, A represents the offset of the sampling point in the upper left corner of E, B represents the offset of the sampling point in the lower left corner of E, C represents the offset of the sampling point in the upper right corner of E, D represents the offset of the sampling point in the lower right corner of E, x represents the absolute value of the difference between C and A on the horizontal axis, y represents the absolute value of the difference between D and C on the vertical axis, dis_x represents the absolute value of the difference between E and A on the horizontal axis, and dis_y represents the absolute value of the difference between E and C on the vertical axis.

[0208] In this specific embodiment, according to the optional solution of storing offsets using a lookup table, the offsets ofst R and ofst B of the red and blue channels corresponding to each pixel point relative to the green channel are interpolated through formula (8). After that, using pos G to represent the coordinate position of the refracted green channel (i.e., the coordinate position of the green channel after offset. It can be understood that the coordinate position of the green channel remains almost unchanged during the refraction process. Therefore, pos G can be represented by the coordinate position before refraction), then the coordinate position of the refracted red channel (i.e., the coordinate position of the red channel after offset) pos R can be calculated through formula (9):

[0209] pos R = pos G + ofst R (9)

[0210] The coordinate position of the refracted blue channel (i.e., the coordinate position of the green channel after offset) is pos B and can be calculated through formula (10):

[0211] pos B = pos G + ofst B (10)

[0212] In this specific embodiment, the edge color fringes can be divided into two parts: one is the color fringes caused by being located at the screen edge; the other is the color fringes introduced by occluding the screen to form a specific shape of the visual field.

[0213] In an alternative solution, the processing flow for eliminating edge color fringes is as Figure 8 shown.

[0214] Step S21: Calculate the offsets of the green and blue channels of the refracted pixel points relative to the red channel;

[0215] In this embodiment, the interpolation results ofst R and ofst B of the coordinate mapping module can be used to obtain the offsets ofst CG and ofst CB of the green and blue channels of the refracted pixel points relative to the red channel.

[0216] Among them, the offset ofst CG of the green channel relative to the red channel can be calculated by formula (11):

[0217] ofst CG = -ofst G (11)

[0218] The offset ofst CB of the blue channel relative to the red channel can be calculated by formula (12):

[0219] ofst CB = ofst B - ofst G (12)

[0220] Step S22: Calculate the coordinate positions of the green and blue channels of the refracted pixel points;

[0221] In this alternative solution, edge color fringe elimination uses the red channel as the reference channel (it is not difficult to understand that the user can use the blue channel or the green channel as the reference channel according to actual needs), and takes the coordinate position of the currently calculated pixel point before refraction as the coordinate position pos CR after the red channel is offset after refraction. Then, the coordinate position pos CG of the green channel after offset after refraction can be calculated by formula (13):

[0222] pos CG = pos CR + ofst CG (13)

[0223] The coordinate position pos of the blue channel after offset after refractionCB It can be calculated by formula (14):

[0224] pos CB = pos CR + ofst CB (14)

[0225] That is, according to the second position offset information, determine the coordinate positions of the channels to be corrected corresponding to each pixel point after offset.

[0226] Step S23: Determine whether the coordinate position of the blue channel of the refracted pixel point exceeds the screen range or is located in the occlusion area. If so, set the blue channel of the pixel point to 0;

[0227] Step S24: Determine whether the coordinate position of the green channel of the refracted pixel point exceeds the screen range or is located in the occlusion area. If so, set the green channel of the pixel point to 0.

[0228] In this optional solution, for each pixel point's pos CG , determine whether it exceeds the screen range and whether it is located in the occlusion area. If it exceeds the screen range or is located in the occlusion area, set the green channel of this pixel point to 0. At the same time, for each pixel point's pos CB , determine whether it exceeds the screen range and whether it is located in the occlusion area. If it exceeds the screen range or is located in the occlusion area, set the blue channel of this pixel point to 0, so that in subsequent pixel generation, the pixel values of the zeroed channels are attenuated in brightness.

[0229] That is, the pixel points whose coordinate positions after offset of the channels to be corrected are located within the edge color border area are used as target pixel points, and the channels to be corrected of the target pixel points are subjected to brightness attenuation processing. The brightness attenuation processing is to set the brightness of the channel to be corrected to zero, or reduce the brightness of the channel to be corrected below a preset brightness value.

[0230] In this specific implementation, the pixel generation module needs to calculate the corresponding pixel values according to the red and blue channel coordinates obtained by formula (9) and formula (10), and assign them to the red channel and blue channel of the corresponding pixel points respectively.

[0231] In an optional solution, the pixel calculation method (that is, pixel interpolation processing) can be bilinear interpolation. Considering that only the red and blue channels need to be generated during pixel generation, and the green channel maintains the original pixel value, interpolating in the non-linear domain will cause brightness loss of the red and blue channels, thus causing color cast. To avoid this problem, this optional solution can first map the pixel values to the linear domain and then perform pixel interpolation processing to avoid brightness loss.

[0232] Specifically, the processing flow of pixel generation is asFigure 9 as shown

[0233] Step S31: Detect whether there are pixel points with a color channel set to 0. If so, perform brightness attenuation processing on the color channel set to 0;

[0234] In this embodiment, the output signal from the edge color fringe elimination module is received. If it is detected that a certain color channel of a certain pixel point is set to 0, the pixel value of that color channel is set to 0, or adjusted below a preset value (i.e., the preset brightness value).

[0235] Step S32: Determine whether brightness compensation is required.

[0236] In this alternative solution, according to the preset configuration information, it is determined whether to generate a brightness compensation instruction (i.e., the target brightness compensation instruction), and after receiving this instruction, brightness compensation is enabled. Otherwise, the pixel values of each pixel point are directly calculated

[0237] Step S33: If brightness compensation is required, map the pixel value of the pixel point from the non-linear domain to the linear domain through the EOTF, perform pixel interpolation processing on the pixel value of the pixel point, and then map the pixel value of the pixel point after brightness compensation from the linear domain back to the non-linear domain through the OETF, and output the pixel value of the pixel point;

[0238] That is, through the processing of the electro-optical transfer function EOTF, the pixel information of each pixel point in the non-linear domain is mapped from the non-linear domain to the linear domain, obtaining the pixel information of each pixel point in the linear domain; perform pixel interpolation processing on the pixel information of each pixel point in the linear domain, obtaining the pixel information of each pixel point in the linear domain after brightness compensation; through the processing of the opto-electronic transfer function OETF, the pixel information of each pixel point in the linear domain after brightness compensation is mapped from the linear domain to the non-linear domain, obtaining the pixel information of each pixel point in the non-linear domain after brightness compensation and outputting it.

[0239] Step S34: If brightness compensation is not required, perform pixel interpolation processing on the pixel value of the pixel point, and output the pixel value of the pixel point.

[0240] After the color difference correction of the above four parts, this specific embodiment can eliminate the edge color fringes caused by lateral color difference in the head-mounted display device.

[0241] Exemplarily, in one example, test Figure 1 such as Figure 10 as shown, in this test Figure 1 there are multiple rectangles arranged in an array. When the color difference correction method provided by this specific embodiment is not applied to this test Figure 1 for processing, a partial enlarged view of this test Figure 1 is as shown in Figure 11As shown, in the local enlarged view of this test Figure 1 , there is an obvious colored edge line type at the edge of the rectangle. After applying the color difference correction method provided in this specific embodiment, as Figure 12 shown, the colored edge at the edge of the rectangle is eliminated.

[0242] In another example, for the test Figure 2 as Figure 13 shown, in this test Figure 13 , there is a circle slightly beyond the field of view. When the color difference correction method provided in this specific embodiment is not applied to process this test Figure 2 , a local enlarged view of this test Figure 2 is as Figure 14 shown. In the local enlarged view of this test Figure 2 , there is an obvious colored edge line type at the arc edge of the circle. After applying the color difference correction method provided in this specific embodiment, as Figure 15 and Figure 16 shown, the colored edge at the arc edge is eliminated.

[0243] In yet another example, in a certain test Figure 3 , there are a large number of densely arranged characters. When the color difference correction method provided in this specific embodiment is not applied to process this test Figure 3 , a local enlarged view of this test Figure 3 is as Figure 17 shown. In the local enlarged view of this test Figure 3 , obvious color separation occurs in the characters. After applying the color difference correction method provided in this specific embodiment, as Figure 18 shown, without brightness compensation, the color separation phenomenon is basically eliminated, but there is an obvious color cast in the characters, and the overall color tends to be green. After brightness compensation, as Figure 19 shown, there is no color cast in the characters, and the color returns to the original character color.

[0244] It should be noted that the above examples are only used to assist in understanding this application and do not constitute a limitation on the color difference correction method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0245] In addition, please refer to Figure 20 , Figure 20 which is a schematic diagram of the device structure of the hardware operating environment involved in the color difference correction method in the embodiments of this application.

[0246] The present application also provides a head-mounted display device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the color difference correction method in the above embodiments.

[0247] Reference is made below to Figure 20 , which shows a schematic structural diagram of a head-mounted display device suitable for implementing the embodiments of the present application. The head-mounted display device in the embodiments of the present application may include, but is not limited to, devices such as MR (Mixed Reality) devices (such as MR glasses, MR helmets), AR (Augmented Reality) devices (such as AR glasses, AR helmets), VR (Virtual Reality) devices (such as VR glasses, VR helmets), XR (Extended Reality) devices (such as XR glasses, XR helmets), or any head-mounted display device capable of implementing the above functions. Figure 20 The head-mounted display device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0248] As Figure 20 shown, the head-mounted display device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can execute various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the head-mounted display device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the head-mounted display device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a head-mounted display device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be alternatively implemented or had.

[0249] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0250] The head-mounted display device provided by the present application adopts the chromatic aberration correction method in the above-mentioned embodiment, and can perform real-time correction processing on lateral chromatic aberration with relatively low hardware complexity and operating load. Compared with the prior art, the beneficial effects of the head-mounted display device provided by the present application are the same as those of the chromatic aberration correction method provided by the above-mentioned embodiment, and other technical features in the head-mounted display device are the same as those disclosed in the method of the above-mentioned embodiment, which will not be elaborated here.

[0251] It should be understood that each part disclosed in the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0252] As described above, it is only the specific implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all of them should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the above-mentioned claims.

[0253] In addition, the present application also provides a computer-readable storage medium, which has computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the steps of the chromatic aberration correction method in the above-mentioned embodiment.

[0254] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0255] The above computer-readable storage medium can be included in a head-mounted display device; it can also exist separately and not be assembled into the head-mounted display device.

[0256] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by the head-mounted display device, the head-mounted display device is caused to: input the pixel sampling point information in the image to be displayed into a pre-constructed target offset model to obtain the first position offset information of the channel to be corrected corresponding to each pixel sampling point information; according to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, and in combination with an interpolation method, interpolate to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed; determine the edge color fringe area of the screen according to the second position offset information, and perform brightness attenuation processing on the edge color fringe area, where the edge color fringe area refers to the color fringe area formed outside the target display area corresponding to the screen display frame.

[0257] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0258] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0259] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0260] The computer-readable storage medium provided by this application stores computer-readable program instructions (i.e., computer programs) for performing the steps of the above-mentioned chromatic aberration correction method, and can achieve real-time correction processing of lateral chromatic aberration with lower hardware complexity and running load. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the chromatic aberration correction method provided by the above embodiments, and will not be elaborated here.

[0261] In addition, an embodiment of the present application further provides a computer program product, including a computer program, which implements the steps of the chromatic aberration correction method in the above embodiments when executed by a processor.

[0262] The computer program product provided by the present application can perform real-time correction processing on lateral chromatic aberration with relatively low hardware complexity and operating load. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as those of the chromatic aberration correction method provided by the above embodiments, and will not be elaborated here.

[0263] The above are only partial embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the specification and drawings of the present application under the technical concept of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for correcting chromatic aberration, characterized in that: The chromatic aberration correction method is applied to a head mounted display device, and the method comprises: Inputting each pixel sampling point information in the image to be displayed into a pre-built target offset model to obtain first position offset information of the channel to be corrected corresponding to each pixel sampling point information; According to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, in combination with the interpolation method, interpolate to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed; According to the second position offset information, the edge color edge area of ​​the screen is determined, and brightness attenuation processing is performed on the edge color edge area, wherein the edge color edge area refers to the color edge area formed outside the calibrated target display area corresponding to the screen display frame.

2. The chromatic aberration correction method according to claim 1, wherein: The step of determining the edge color edge area of ​​the screen according to the second position offset information comprises: According to the second position offset information, determine the coordinate position of the to-be-corrected channel offset corresponding to each pixel point; Based on the coordinate position of the to-be-corrected channel corresponding to each pixel after offset, the edge color fringing area of ​​the screen is determined.

3. The chromatic aberration correction method according to claim 2, wherein: The step of performing brightness attenuation processing on the edge color fringing area comprises: The pixel point whose coordinate position after the offset of the channel to be corrected is located in the edge color fringing area is taken as the target pixel point; A brightness attenuation process is performed on the channel to be corrected of the target pixel point, wherein the brightness attenuation process is to set the brightness of the channel to be corrected to zero.

4. The chromatic aberration correction method according to claim 1, wherein: The method further comprises: The channel to be corrected corresponding to each pixel point is corrected according to the second position offset information.

5. The chromatic aberration correction method according to claim 4, characterized in that: After the step of correcting the to-be-corrected channel corresponding to each pixel point according to the second position offset information, the method further includes: By processing the electro-optic transfer function EOTF, the pixel information of each pixel point in the nonlinear domain is mapped from the nonlinear domain to the linear domain, so as to obtain the pixel information of each pixel point in the linear domain; Performing pixel interpolation processing on the pixel information of each pixel point in the linear domain to obtain the pixel information of each pixel point in the linear domain after brightness compensation; Through the processing of the photoelectric transfer function OETF, the pixel information in the linear domain after the brightness compensation of each pixel point is mapped from the linear domain to the nonlinear domain, and the pixel information in the nonlinear domain after the brightness compensation of each pixel point is obtained and output.

6. The chromatic aberration correction method according to claim 5, characterized in that: The method further comprises: When the target brightness compensation instruction is received, the following step is performed: mapping the pixel information of each pixel point in the nonlinear domain from the nonlinear domain to the linear domain through the processing of the electro-optical transfer function EOTF.

7. The chromatic aberration correction method according to any one of claims 1 to 6, characterized in that: Before inputting the information of each pixel sampling point in the image to be displayed into the pre-built target offset model, the method further includes: Acquire display module parameter information of the head mounted display device, wherein the display module parameter information at least includes light transfer parameters and screen module design parameters; A chromatic aberration offset model of the head mounted display device is constructed according to the display module parameter information to obtain a pre-constructed target offset model.

8. The color difference correction method according to claim 7, characterized in that: The step of constructing a color difference offset model of the head mounted display device according to the display module parameter information includes: Collecting a target dot matrix virtual image formed after the target dot matrix light passes through the lens of the head mounted display device; A chromatic aberration offset model of the head mounted display device is constructed based on the target dot matrix virtual image and the display module parameter information.

9. The chromatic aberration correction method according to claim 8, characterized in that: The step of constructing a chromatic aberration offset model of the head mounted display device based on the target dot matrix virtual image and the display module parameter information comprises: Acquire a standard dot matrix image corresponding to the target dot matrix light, perform image processing on the standard dot matrix image to obtain first feature information of each pixel position in the standard dot matrix image, and perform image processing on the target dot matrix virtual image to obtain second feature information of each pixel position in the target dot matrix virtual image; Comparing the first feature information with the second feature information to obtain initial pixel color difference information at each pixel position between the standard dot matrix image and the target dot matrix virtual image; Based on the display module parameter information, the initial pixel color difference information is corrected to obtain corrected target pixel color difference information of each pixel position; Based on the corrected target pixel color difference information at each pixel position, a color difference offset model of the head mounted display device is constructed.

10. The chromatic aberration correction method according to claim 9, wherein: The step of constructing a color difference offset model of the head mounted display device based on the corrected target pixel color difference information of each pixel position includes: Based on the corrected target pixel color difference information of each pixel position, determining an offset of a channel to be corrected relative to a reference channel in the target pixel color difference information; According to the offset of the channel to be corrected in the target pixel color difference information relative to the reference channel, a distortion model of the channel to be corrected in the target pixel color difference information relative to the reference channel is constructed, Based on the distortion model, a chromatic aberration offset model of the head mounted display device is constructed.

11. A chromatic aberration correction method, the chromatic aberration correction method being applied to a head mounted display device, the method comprising: Based on a pre-built chromatic aberration correction lookup table, first position offset information of the channel to be corrected corresponding to each pixel sampling point information in the image to be displayed is obtained; According to the first position offset information of the channel to be corrected corresponding to each pixel sampling point information, in combination with the interpolation method, interpolate to obtain the second position offset information of the channel to be corrected corresponding to each pixel point on the image to be displayed; According to the second position offset information, the edge color edge area of ​​the screen is determined, and brightness attenuation processing is performed on the edge color edge area, wherein the edge color edge area refers to the color edge area formed outside the calibrated target display area corresponding to the screen display frame.

12. A head mounted display device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the chromatic aberration correction method according to any one of claims 1 to 10 or 11 is implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the chromatic aberration correction method according to any one of claims 1 to 10 or 11 is implemented.

Images