Image correction method and apparatus, display device, storage medium, and program product

By obtaining the center point coordinates and distortion coefficients of the test picture when the display device is started and calculating the distorted pixel coordinates, the problem of the inability to uniformly correct the optical distortion of different lenses in the traditional method is solved, and the tailor-made for each display device is achieved Optimize image distortion correction to improve the accuracy of image correction.

WO2024244053A1PCT designated stage expired Publication Date: 2024-12-05INTERFACE ADVANCED TECH (CHENGDU) CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2023/099854
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-01
Filing Date
2023-06-13
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

In the traditional way, the correction method for image deformation caused by optical distortion of lenses is to set the distortion function during the development stage of the display device, which cannot adapt to the optical distortion degree of different lenses, resulting in the inability to uniformly correct the image distortion in each display device.

Method used

When the display device is started, by obtaining the center point coordinates of the test picture, multiple distortion variables and their corresponding distortion coefficients are determined, the target sampling point is selected, the distorted pixel point coordinates are calculated, the actual shape variable is calculated, and the actual shape is based on the actual shape. Variables correct the test image.

Benefits of technology

It realizes the precise and effective image distortion correction of each display device, adapts to the optical distortion degree of different lenses, and improves the accuracy and effect of image correction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023099854_05122024_PF_FP_ABST
    Figure CN2023099854_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to an image correction method and apparatus, a head-mounted display device, a storage medium and a computer program product. The method comprises: in the case that a display device is starting up, acquiring a currently displayed test picture (202); calculating the central point coordinates of the test picture (204); determining a plurality of distortion quantities and a distortion coefficient corresponding to each distortion quantity (206); acquiring the original pixel point coordinates of target sampling points (208); according to the plurality of distortion quantities, the distortion coefficient corresponding to each distortion quantity and the original pixel point coordinates, calculating the pixel point coordinates of the target sampling points after distortion (210); calculating an actual deformation quantity of the test picture (212); and according to the actual deformation quantity, correcting the test picture (214).
Need to check novelty before this filing date? Find Prior Art

Description

Image correction method, device, display device, storage medium and program product

[0001] This application claims priority to Chinese patent application number 2023106445949, filed with the China Patent Office on June 1, 2023, entitled “Image Correction Method, Device, Display Device, Storage Medium and Program Product,” the entire text of which is hereby incorporated by reference. Technical Field

[0002] The present application relates to the field of image processing technology, and in particular to an image correction method, apparatus, display device, storage medium, and computer program product. Background Art

[0003] Traditionally, image distortion caused by lens optical distortion is corrected during the display device development phase. This involves first measuring the optical distortion offset data of each sub-pixel. Based on this measured optical distortion offset data, a corresponding distortion function is then calculated and applied to the system to correct the image distortion.

[0004] However, because the distortion function is set on the system side during the display device development phase, and the degree of optical distortion of each lens in the display device varies, it is impossible to correct the image in each display device using a single distortion function.

[0005] Summary of the Invention

[0006] Based on this, according to various embodiments of the present application, an image correction method, apparatus, head-mounted display device, computer-readable storage medium, and computer program product are provided.

[0007] In a first aspect, the present application provides an image correction method. Applied to a display device, the method comprises:

[0008] When the display device is starting up, obtaining a currently displayed test image;

[0009] Calculating the center point coordinates of the test image;

[0010] Determining, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts;

[0011] Selecting a target sampling point and obtaining the original pixel coordinates of the target sampling point; the target sampling point is a vertex of each boundary of the test image;

[0012] Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates;

[0013] Calculating the actual deformation of the test image according to the distorted pixel coordinates;

[0014] The test image is corrected according to the actual deformation amount.

[0015] In one embodiment, the test image includes a preset undistorted rectangular area, and the calculating the current center point coordinates of the test image includes:

[0016] The current center point coordinates of the test image are calculated according to the midpoints of the four sides of the preset undistorted rectangular area.

[0017] In one embodiment, determining a plurality of distortion amounts and a distortion coefficient corresponding to each distortion amount according to the center point coordinates includes:

[0018] Dividing the test image into a plurality of concentric regions according to a horizontal axis, a vertical axis, and the center point coordinates, wherein the center point coordinates of each region are consistent with the center point coordinates of the test image; the plurality of concentric regions include a preset undistorted rectangular region, and the horizontal axis and the vertical axis are set based on the center point coordinates of the test image;

[0019] Calculating, based on the center point coordinates, a first distortion amount and a residual distortion amount divided by the first distortion amount, where a distortion coefficient corresponding to the first distortion amount is a preset first distortion coefficient, and the first distortion amount is a vertical distance from the center point coordinates to a boundary of the preset undistorted rectangular area;

[0020] determining, according to the first distortion amount and a residual distortion amount obtained by dividing the first distortion amount, a distortion coefficient corresponding to the residual distortion amount;

[0021] Determining whether the distortion coefficient needs to be adjusted according to preset requirements;

[0022] When the distortion coefficient needs to be adjusted, the distortion coefficient is adjusted to obtain an adjusted distortion coefficient.

[0023] In one embodiment, determining whether the distortion coefficient needs to be adjusted according to a preset requirement includes:

[0024] According to the coordinates of the center point of the test image, each area in the divided test image is assigned an X-axis and a Y-axis, and a sampling point is set for each area;

[0025] Based on the sampling points in each area, it is determined whether the distortion coefficient needs to be adjusted.

[0026] In one embodiment, the sampling points include first-type sampling points, where the first-type sampling points are intersections of an X-axis set in each region and a boundary of the corresponding region; the distortion coefficient includes a horizontal distortion coefficient; and determining whether the distortion coefficient needs to be adjusted based on the sampling points in each region includes:

[0027] Determine whether the first-class sampling points in each area are on the same horizontal line;

[0028] When the first type of sampling points are not located on the same horizontal line, it is determined that the horizontal distortion coefficient needs to be adjusted.

[0029] In one embodiment, the sampling points include second-type sampling points, where the second-type sampling points are intersections of a Y-axis set in each region and a boundary of the corresponding region; the distortion coefficient includes a vertical distortion coefficient; and determining whether the distortion coefficient needs to be adjusted based on the sampling points in each region further includes:

[0030] Determine whether the second type of sampling points in each area are on the same vertical line;

[0031] When the second-type sampling points are not located on the same vertical line, it is determined that the vertical distortion coefficient needs to be adjusted.

[0032] In one embodiment, determining whether the distortion coefficient needs to be adjusted based on the sampling points in each area further includes:

[0033] When the first-type sampling points are on the same horizontal line and the second-type sampling points are on the same vertical line, it is determined that the distortion coefficient does not need to be adjusted.

[0034] In a second aspect, the present application further provides an image correction device. Applied to a display device, the device comprises:

[0035] An image acquisition module, configured to acquire a currently displayed test image when the display device is starting up;

[0036] A first coordinate calculation module, used to calculate the coordinates of the center point of the test image;

[0037] a coefficient determination module, configured to determine a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts according to the coordinates of the center point;

[0038] A second coordinate calculation module is used to select a target sampling point and obtain the original pixel coordinates of the target sampling point; the target sampling point is the vertex of each boundary of the test image;

[0039] a deformation variable calculation module, configured to calculate the distorted pixel coordinates of the target sampling point based on a plurality of distortion variables, a distortion coefficient corresponding to each of the distortion variables, and the original pixel coordinates;

[0040] A shape variable calculation module, configured to calculate the actual shape variable of the test image based on the distorted pixel coordinates;

[0041] A correction module is used to correct the test image according to the actual deformation amount.

[0042] In a third aspect, the present application further provides a head-mounted display device. The head-mounted display device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0043] When the display device is starting up, obtaining a currently displayed test image;

[0044] Calculating the center point coordinates of the test image;

[0045] Determining, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts;

[0046] Selecting a target sampling point and obtaining the original pixel coordinates of the target sampling point; the target sampling point is a vertex of each boundary of the test image;

[0047] Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates;

[0048] Calculating the actual deformation of the test image according to the distorted pixel coordinates;

[0049] The test image is corrected according to the actual deformation amount.

[0050] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0051] When the display device is starting up, obtaining a currently displayed test image;

[0052] Calculating the center point coordinates of the test image;

[0053] Determining, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts;

[0054] Selecting a target sampling point and obtaining the original pixel coordinates of the target sampling point; the target sampling point is a vertex of each boundary of the test image;

[0055] Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates;

[0056] Calculating the actual deformation of the test image according to the distorted pixel coordinates;

[0057] The test image is corrected according to the actual deformation amount.

[0058] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0059] When the display device is starting up, obtaining a currently displayed test image;

[0060] Calculating the center point coordinates of the test image;

[0061] Determining, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts;

[0062] Selecting a target sampling point and obtaining the original pixel coordinates of the target sampling point; the target sampling point is a vertex of each boundary of the test image;

[0063] Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates;

[0064] Calculating the actual deformation of the test image according to the distorted pixel coordinates;

[0065] The test image is corrected according to the actual deformation amount.

[0066] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below, and other features, objects, and advantages of the present application will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0068] FIG1 is a diagram illustrating an application environment of an image correction method according to one or more embodiments;

[0069] FIG2 is a flow chart of an image correction method according to one or more embodiments;

[0070] FIG3 is a structural diagram of an image correction module according to one or more embodiments;

[0071] FIG4 is a schematic diagram of an undistorted image according to one or more embodiments;

[0072] FIG5 is a schematic diagram of a barrel-distorted image according to one or more embodiments;

[0073] FIG6 is a schematic diagram of an image with pincushion distortion according to one or more embodiments;

[0074] FIG7 is a schematic diagram of calculating an actual deformation amount according to one or more embodiments;

[0075] FIG8 is a schematic diagram of calculating center point coordinates according to one or more embodiments;

[0076] FIG9 is a schematic diagram of a process for determining distortion coefficients according to one or more embodiments;

[0077] FIG10 is a schematic diagram of the division of concentric regions according to one or more embodiments;

[0078] FIG11 is a schematic diagram of a distortion coefficient adjustment process according to one or more embodiments;

[0079] FIG12 is a diagram illustrating horizontal distortion coefficients and vertical distortion coefficients according to one or more embodiments;

[0080] FIG13 is a schematic diagram of a process of using a display device according to one or more embodiments;

[0081] FIG14 is a flow chart of an image correction method according to another embodiment;

[0082] FIG15 is a block diagram of an image correction device according to one or more embodiments;

[0083] FIG16 is a diagram illustrating the internal structure of a head-mounted display device according to one or more embodiments. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0085] With the advancement of technology, more and more people are using head-mounted displays (HMDs), such as VR monitors, to view images. However, when viewing images through these displays, some degree of distortion occurs. Traditionally, image distortion caused by lens optical distortion is corrected by setting a distortion function during the display device development phase. However, the degree of optical distortion of lenses used in display devices varies, making it impossible to correct images for every display device using a single distortion function.

[0086] To address this issue, the image correction method of the present application is proposed and can be applied in the application environment shown in Figure 1. Display device 102 communicates with server 104 via a network. A data storage system can store images that server 104 needs to process. The data storage system can be integrated with server 104 or located in the cloud or on another network server. Display device 102 can be equipped with a memory for storing images that display device 102 needs to process.

[0087] When the display device 102 is being started, it obtains a test image sent by the server 104, calculates the center point coordinates of the test image based on the test image, and determines multiple distortion amounts and the distortion coefficient corresponding to each distortion amount based on the center point coordinates; selects a target sampling point and obtains the original pixel coordinates of the target sampling point; the target sampling point is the vertex of each boundary of the test image; calculates the distorted pixel coordinates of the target sampling point based on the multiple distortion amounts, the distortion coefficient corresponding to each distortion amount, and the original pixel coordinates; calculates the actual deformation amount of the test image based on the distorted pixel coordinates; and corrects the test image based on the actual deformation amount.

[0088] The test image may be stored in the server 104 or pre-stored in the memory of the display device 102 .

[0089] The display device 102 includes but is not limited to various portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0090] In one embodiment, as shown in FIG2 , an image correction method is provided, which is applied to the display device 102 in FIG1 as an example for illustration, and includes the following steps:

[0091] Step 202: When the display device is starting up, obtain a currently displayed test image.

[0092] In some embodiments, the display device may be a head-mounted display device, which includes an image correction module. Specifically, the structure of the image correction module can be referred to FIG3 , which includes an image reflection unit 31, an eye tracking unit 32, an optical display unit 33, and a computing unit 34. The optical display unit 33 includes an optical lens 331, a display screen 332,

[0093] The image reflection unit 31 includes two parts: a mirror surface and an opaque, scratch-resistant material. It can be designed with a pull-out protective cover. When the head-mounted display device is not turned on, it can be used to protect the optical lens and display screen in the optical display unit. When the head-mounted display device is turned on, it is used to reflect the image from the display screen to provide the eye tracking unit with distortion information of the test image and send it to the computing unit to achieve image correction.

[0094] The eye tracking unit 32 includes a camera, which detects distortion information of the test image and is used to calculate the user's eye rotation angle and gaze direction.

[0095] The optical display unit 33 includes an optical lens 331 and a display screen 332 , and performs near-eye display through the combination of the optical lens and the display screen.

[0096] The computing unit 34 is used to perform image correction or estimate the user's gaze point position based on the information detected by the eye tracking unit.

[0097] A test image is set on the display device. The test image has a collinear feature. When the test image is displayed normally, the midpoints of the four sides of the test image and the two vertices of the corresponding boundaries are on a straight line, indicating that the test image has a collinear feature. Specifically, the display of the test image in an undistorted state can be referred to in Figure 4.

[0098] For example, when the head mounted display device is starting up, a currently displayed test image may be obtained from a server or a memory of the head mounted display device. The currently displayed test image may be in a distorted state or in an undistorted state.

[0099] Image distortion is a type of optical distortion, which refers to an optical aberration that causes physical straight lines to distort and bend. Image distortion generally includes two types: barrel distortion and pincushion distortion. Barrel distortion is a lens-induced distortion that causes the image to expand in a barrel shape, often seen with wide-angle lenses. Pincushion distortion is a lens-induced "contraction" of the image toward the center, often seen with telephoto lenses. For examples of barrel distortion, see Figure 5; for pincushion distortion, see Figure 6.

[0100] Step 204: Calculate the center point coordinates of the test image.

[0101] The coordinates of the center point of the test picture are the center of the test image.

[0102] In some embodiments, the eye tracking unit shown in FIG3 can be used to monitor a test image to calculate the coordinates of the center point of the test image. Specifically, if the test image is distorted, the center coordinates of the test image can be calculated by setting an undistorted rectangular area in the test image.

[0103] Step 206: Determine multiple distortion amounts and a distortion coefficient corresponding to each distortion amount according to the center point coordinates.

[0104] Exemplarily, the distortion amount is calculated based on the coordinates of the center point, and the distortion coefficient corresponding to each distortion amount can be determined based on the sizes of the multiple distortion amounts.

[0105] In some embodiments, the test image can be divided into multiple concentric areas, and the corresponding centers are the center point coordinates of the test image. Specifically, the areas can be divided outward in sequence based on the center point coordinates of the test image to obtain the distortion amount of each area. Based on the distortion amount of each area, the distortion coefficient corresponding to each distortion amount is determined.

[0106] Step 208 : Select target sampling points and obtain original pixel coordinates of the target sampling points; the target sampling points are the boundary vertices of the test image.

[0107] For example, target sampling points are selected from a test image, and vertices of each boundary in the test image can be selected. Generally, the vertices of each boundary in the test image include four vertices of the test image and four center points of the test image boundary. Therefore, there are a total of eight target sampling points. For more specific information on the locations of the target sampling points on the test image, see Figure 7.

[0108] In some embodiments, after the target sampling point is selected, the original pixel coordinates of the target sampling point may be obtained, that is, the coordinates of the target sampling point without distortion.

[0109] Step 210 : Calculate the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, the distortion coefficient corresponding to each distortion amount, and the original pixel coordinates.

[0110] In some embodiments, the test image can be divided into multiple concentric areas, and the corresponding centers are the center coordinates of the test image. When there are as many concentric areas as there are distortion amounts, and the distortion coefficients corresponding to each distortion amount, the pixel coordinates of the target sampling point after distortion are calculated based on the multiple distortion amounts, the distortion coefficients corresponding to each distortion amount, and the original pixel coordinates of the selected target sampling point.

[0111] Step 212: Calculate the actual deformation of the test image based on the distorted pixel coordinates.

[0112] In some embodiments, the actual deformation of the test image can be calculated based on the pixel coordinates of the distorted target sampling points. Specifically, the target sampling points include the vertices of the test image at each boundary. The actual deformation of the test image can be calculated based on the vertical distance between the highest point of the test image at the boundary and the lowest point of the corresponding boundary.

[0113] In order to better understand how to calculate the actual deformation, an example is given to illustrate. Specifically, referring to FIG7, a schematic diagram of calculating the actual deformation is shown, which is a method of calculating the actual deformation of a test image under the condition of barrel distortion. There are four actual deformations in the test image, namely, d T d D d L and d R , if the X-axis and Y-axis are divided according to the center coordinates of the test image, the corresponding d T and d D It is the difference value of the test image on the Y axis, which can be referred to as Y d ; corresponding d L and d R Is the difference value of the test image on the X axis, which can be referred to as X d .

[0114] Specifically, the actual deformation amount may be calculated based on the vertical distance between the highest point on the boundary of the test image and the lowest point on the corresponding boundary.

[0115] Step 214: Correct the test image according to the actual deformation amount.

[0116] Exemplarily, when the magnitude of the actual deformation amount is equal to 0 or close to 0, it is determined that the test image is in an undistorted state; when the magnitude of the actual deformation amount is greater than 0, it is determined that the test image is in a distorted state.

[0117] In some embodiments, after the actual deformation amount of the test image is calculated, the image distortion compensation can be performed by the image processing chip in the display device to correct the X d and Y d , until the actual deformation is equal to 0 or close to 0.

[0118] Exemplarily, the image processing chip may be a sensor chip or a video processing chip. Specifically, the type of the image processing chip may be selected according to actual conditions, and this application does not limit this.

[0119] In the above-mentioned image correction method, when the display device is starting up, a currently displayed test image is obtained; the coordinates of the center point of the test image are calculated; based on the center point coordinates, multiple distortion amounts and the distortion coefficient corresponding to each distortion amount are determined; target sampling points are selected and the original pixel coordinates of the target sampling points are obtained; the target sampling points are vertices of each boundary of the test image; based on the multiple distortion amounts, the distortion coefficient corresponding to each distortion amount, and the original pixel coordinates, the distorted pixel coordinates of the target sampling points are calculated; based on the distorted pixel coordinates, the actual deformation amount of the test image is calculated; and based on the actual deformation amount, the test image is corrected. Using this method, when the display device is starting up, the actual deformation amount of the test image can be calculated based on the currently displayed test image, and the test image can be corrected based on the actual deformation amount, thereby achieving customized image distortion optimization for each display device.

[0120] In one embodiment, the test image includes a preset undistorted rectangular area, and step 104 includes:

[0121] The current center point coordinates of the test image are calculated according to the midpoints of the four sides of the preset undistorted rectangular area.

[0122] In some embodiments, regardless of whether the test image has barrel-shaped or pincushion-shaped optical distortion, the farther away from the center of the lens, the more severe the image distortion. Therefore, in the case of barrel distortion and pincushion distortion, the distortion of the sampling point closer to the center point is smaller. Based on this, the test image is set to include a preset undistorted rectangular area. Specifically, refer to Figure 7, which shows a schematic diagram of calculating the coordinates of the center point. When the test image has barrel distortion, the preset undistorted rectangular area may include P A 、P B 、P C 、P D 、P E These five points. Among them, PA 、P B 、P D 、P E is the boundary center of the undistorted rectangular area, P C is the center point coordinate. Considering the varying characteristics of optical lenses, the size of this rectangular area is not specifically defined. Specifically, a smaller rectangular area results in a more refined image distortion adjustment, but also requires more image data processing. Conversely, a larger rectangular area results in a coarser image distortion adjustment, but requires less image data processing.

[0123] In this embodiment, the current center point coordinates of the test image can be calculated based on the midpoints of the four sides of the preset undistorted rectangular area, and then the actual deformation of the test image can be calculated based on the center point coordinates. The test image can be corrected according to the actual deformation, thereby achieving customized optimization of image distortion for each display device.

[0124] In one embodiment, referring to FIG9 , a schematic diagram of a process for determining a distortion coefficient is shown, where step 208 includes:

[0125] Step 902: Divide the test image into multiple concentric areas according to the horizontal axis, the vertical axis, and the center point coordinates, wherein the center point coordinates of each area are consistent with the center point coordinates of the test image; the multiple concentric areas include a preset undistorted rectangular area, and the horizontal axis and the vertical axis are set based on the center point coordinates of the test image.

[0126] In some embodiments, based on the center point coordinates of the test image, a horizontal axis and a vertical axis are set to divide the test image into a plurality of concentric regions according to the center point coordinates, the horizontal axis, and the vertical axis of the test image. Specifically, the regions into which the test image is divided may include a preset undistorted rectangular region, and the test image may be divided into a preset number of concentric regions based on the size of the preset undistorted rectangular region. Specifically, the division of the concentric regions in the test image can refer to FIG. 10 , including the center point coordinates P0 and four concentric regions, including region r0, region r1, region r2, and region r3.

[0127] Illustratively, the more areas the test image is divided into, the finer the areas of image distortion adjustment are, but the greater the workload for image data processing. Specifically, the size of the preset number can be set according to actual conditions, and this application does not limit it here.

[0128] Specifically, in the process of dividing the concentric areas in the test image, the size of each area can be set to be the same, or it can be set according to actual conditions, which is not limited in this application.

[0129] In step 904, a first distortion amount and a residual distortion amount after dividing the first distortion amount are calculated based on the center point coordinates. The distortion coefficient corresponding to the first distortion amount is a preset first distortion coefficient. The first distortion amount is a vertical distance from the center point coordinates to a boundary of the preset undistorted rectangular area.

[0130] In some embodiments, a first distortion value and a residual distortion value after subtracting the first distortion value can be calculated based on the center point coordinates of the test image. The first distortion value is the vertical distance from the center point coordinates to the boundary of a preset undistorted rectangular area. Since the preset undistorted rectangle is rectangular, the corresponding first distortion value has two values. The residual distortion value is the vertical distance from the center point coordinates to the boundary of an area remaining from the preset undistorted rectangular area. Specifically, since the first distortion value is the vertical distance from the center point coordinates to the boundary of the preset undistorted rectangular area, the first distortion value is determined to be a baseline value, and the corresponding distortion coefficient is a preset first distortion coefficient. The preset first distortion coefficient can be selected based on empirical values, and this application does not limit this.

[0131] Step 906 : Determine a distortion coefficient corresponding to the residual distortion amount according to the first distortion amount and the residual distortion amount after dividing the first distortion amount.

[0132] In some embodiments, the first distortion amount may be a reference value, and a distortion coefficient corresponding to the residual distortion amount may be calculated based on the first distortion amount and the residual distortion amount after dividing the first distortion amount. The distortion coefficient may be obtained by dividing the residual distortion amount by the first distortion amount.

[0133] To better understand how distortion coefficients are calculated, let's use an example. For a test image, four distortion variables are present: first distortion r0, second distortion r1, third distortion r2, and fourth distortion r3. The distortion coefficient k1 for the second distortion r1 is calculated as k1 = r1 ÷ r0; the distortion coefficient k2 for the third distortion r2 is calculated as k2 = r2 ÷ r0; and the distortion coefficient k3 for the fourth distortion r3 is calculated as k3 = r3 ÷ r0.

[0134] Step 908: Determine whether the distortion coefficient needs to be adjusted according to preset requirements.

[0135] In some embodiments, whether the distortion coefficient needs to be adjusted may be determined based on preset requirements. Specifically, the preset requirements may be setting sampling points, and whether the distortion coefficient needs to be adjusted may be determined based on coordinate information of the set sampling points.

[0136] Step 910: If the distortion coefficient needs to be adjusted, adjust the distortion coefficient to obtain an adjusted distortion coefficient.

[0137] In some embodiments, when it is determined that the distortion coefficient needs to be adjusted, the distortion coefficient is adjusted to obtain an adjusted distortion coefficient until the adjusted distortion coefficient meets a preset requirement. The distortion coefficient is then no longer adjusted, and the pixel coordinates of the target sampling point after distortion are calculated based on the adjusted distortion coefficient.

[0138] In this embodiment, the first distortion amount and the residual distortion amount are calculated based on the center point coordinates, and the distortion coefficient corresponding to the residual distortion amount is determined based on the first distortion amount and the residual distortion amount. Based on preset requirements, it is determined whether the distortion coefficient needs to be adjusted, thereby achieving more precise adjustment of the distortion coefficient, thereby achieving more precise image correction based on the distortion coefficient.

[0139] In one embodiment, step 906 includes:

[0140] According to the coordinates of the center point of the test image, each area in the divided test image is assigned an X-axis and a Y-axis, and a sampling point is set for each area;

[0141] Based on the sampling points in each area, it is determined whether the distortion coefficient needs to be adjusted.

[0142] In some embodiments, the sampling point setting method in each area can be to set the X-axis and Y-axis of each concentric area in the test image based on the coordinates of the center point of the test image as the origin to obtain the sampling points of each area, so as to determine whether the distortion coefficient needs to be adjusted based on the sampling points of each area.

[0143] In this embodiment, based on the center point coordinates of the test image, the horizontal axis and the vertical axis are set to divide the test image into multiple concentric areas according to the center point coordinates, the horizontal axis and the vertical axis of the test image. According to the center point coordinates of the test image, the X axis and the Y axis are set for each area in the divided test image, and the sampling points of each area are set. According to the sampling points on each area, it is determined whether the distortion coefficient needs to be adjusted. By dividing the test image into multiple concentric areas and setting the sampling points of each area, it is possible to more accurately determine whether the distortion coefficient needs to be adjusted based on the sampling points of each area of ​​the test image.

[0144] In one embodiment, the sampling points include first-type sampling points, where the first-type sampling points are intersections of an X-axis set in each region and a boundary of the corresponding region; the distortion coefficient includes a horizontal distortion coefficient; and determining whether the distortion coefficient needs to be adjusted based on the sampling points in each region includes:

[0145] Determine whether the first-class sampling points in each area are on the same horizontal line;

[0146] When the first type of sampling points are not located on the same horizontal line, it is determined that the horizontal distortion coefficient needs to be adjusted.

[0147] The distortion amount may include a horizontal distortion amount, and the corresponding distortion coefficient includes a horizontal distortion coefficient. The sampling points may include first-category sampling points, where the first-category sampling points are the intersections of an X-axis defined for each region and the boundary of the corresponding region. Specifically, a determination is made as to whether the first-category sampling points of each region are on the same horizontal line. If the first-category sampling points of each region in the test image are not on the same horizontal line, it is determined that the horizontal distortion coefficient needs to be adjusted. Specifically, the horizontal distortion coefficient may be adjusted based on empirical values ​​until the first-category sampling points of each region are on the same horizontal line after adjustment.

[0148] In this embodiment, the intersection of the x-axis set in each area and the boundary of the corresponding area is selected as the first type of sampling point, and it is determined whether the first type of sampling points are on the same horizontal line. If the first type of sampling points are not on the same horizontal line, the horizontal distortion coefficient is adjusted to obtain a more accurate horizontal distortion coefficient, and more accurate image correction is achieved based on the more accurate horizontal distortion coefficient.

[0149] In one embodiment, the sampling points include second-type sampling points, where the second-type sampling points are intersections of a Y-axis set in each region and a boundary of the corresponding region; the distortion coefficient includes a vertical distortion coefficient; and determining whether the distortion coefficient needs to be adjusted based on the sampling points in each region further includes:

[0150] Determine whether the second type of sampling points in each area are on the same vertical line;

[0151] When the second-type sampling points are not located on the same vertical line, it is determined that the vertical distortion coefficient needs to be adjusted.

[0152] The distortion amount may include a vertical distortion amount, and the corresponding distortion coefficient includes a vertical distortion coefficient. The sampling points may include second-type sampling points, where the second-type sampling points are the intersections of a Y-axis defined for each region and the boundary of the corresponding region. Specifically, a determination is made as to whether the second-type sampling points of each region are on the same vertical line. If the second-type sampling points of each region in the test image are not on the same vertical line, it is determined that the vertical distortion coefficient needs to be adjusted. Specifically, the vertical distortion coefficient may be adjusted based on empirical values ​​until the second-type sampling points of each region are on the same vertical line after adjustment.

[0153] In this embodiment, the intersection of the Y-axis set in each area and the boundary of the corresponding area is selected as the second type of sampling point, and it is determined whether the second type of sampling points are on the same vertical line. If the second type of sampling points are not on the same vertical line, the vertical distortion coefficient is adjusted to obtain a more accurate vertical distortion coefficient. Based on the more accurate vertical distortion coefficient, more accurate image correction is achieved.

[0154] In one embodiment, determining whether the distortion coefficient needs to be adjusted based on the sampling points in each area further includes:

[0155] When the first-type sampling points are on the same horizontal line and the second-type sampling points are on the same vertical line, it is determined that the distortion coefficient does not need to be adjusted.

[0156] In some embodiments, if the first and second sampling points in the test image meet preset requirements, it is determined that the distortion coefficient does not need to be adjusted. Specifically, when the distortion coefficient needs to be adjusted, after the distortion coefficient is adjusted to obtain the adjusted distortion coefficient, it is necessary to again determine whether the first sampling points are on the same horizontal line and whether the second sampling points are on the same vertical line, until the first sampling points corresponding to the adjusted distortion coefficient are on the same horizontal line and the second sampling points are on the same vertical line.

[0157] In this embodiment, when the first type of sampling points are on the same horizontal line and the second type of sampling points are on the same vertical line, it means that there is no need to adjust the distortion coefficient. The distorted pixel coordinates of the target sampling point are calculated based on multiple distortion amounts, the distortion coefficient corresponding to each distortion amount, and the original pixel coordinates, thereby calculating the actual deformation amount of the test image. The test image is then corrected based on the actual deformation amount, achieving customized image distortion optimization for each display device.

[0158] In order to better understand the adjustment process of the distortion coefficient, an example is used to illustrate it. Referring to FIG11 , a schematic diagram of the adjustment process of the distortion coefficient is shown.

[0159] Step 1102: When the display device is starting up, obtain the currently displayed test image.

[0160] Step 1104: Calculate the center coordinates of the test image.

[0161] Step 1106 : Divide the test image into multiple concentric areas, select the first type of sampling points and the second type of sampling points in each area, and calculate the coordinates of the sampling points.

[0162] Specifically, referring to FIG12, a schematic diagram of the horizontal distortion coefficient and the vertical distortion coefficient is shown. Taking the test image as divided into four concentric areas as an example, the middle area is a preset undistorted rectangular area, and the preset first distortion coefficient includes the first horizontal distortion coefficient and the first vertical distortion coefficient The distortion coefficients corresponding to the remaining distortion are the second distortion coefficient, the third distortion coefficient, and the fourth distortion coefficient; the second distortion coefficient includes the second horizontal distortion coefficient and the second vertical distortion coefficient The third distortion coefficient includes a third horizontal distortion coefficient and the first vertical distortion coefficient The fourth distortion coefficient includes the third horizontal distortion coefficient and the first vertical distortion coefficient The distorted pixel coordinates of the sampling point are calculated according to the original pixel coordinates of the sampling point, multiple distortion amounts, and corresponding distortion coefficients.

[0163] Specifically, taking four concentric areas as an example, the formula for calculating the distorted pixel coordinates of the sampling point based on the original pixel coordinates of the sampling point, multiple distortion amounts, and corresponding distortion coefficients is as follows:

[0164] Among them, (X u ,X u ) is the original pixel coordinate, (X d ,X d ) is the pixel coordinate after distortion, k offset-x is the compensation adjustment value for the X axis, k offset-y The compensation adjustment value for the Y axis can be set based on experience. Generally, the set value is too small or is set to 0.

[0165] Step 1108 , determining whether the first type of sampling points in each area are on the same horizontal line.

[0166] Step 1110: If the first type of sampling points in each area are not on the same horizontal line, adjust the horizontal distortion coefficient of each area.

[0167] Step 1112: Determine whether the second-type sampling points in each area are on the same vertical line.

[0168] Step 1114: If the second type of sampling points in each area are not on the same vertical line, adjust the vertical distortion coefficient of each area.

[0169] Step 1116 determines whether the distortion coefficients for each region have been adjusted. If not, the first-category sampling points corresponding to the distortion coefficients are on the same horizontal line, and the second-category sampling points are on the same vertical line. If the distortion coefficients for each region have been adjusted, the coordinates of the sampling points for each region are recalculated based on the adjusted distortion coefficients.

[0170] In order to better understand the adjustment process of the distortion coefficient, an example is given for illustration. Referring to FIG13 , a schematic diagram of a flow chart of using a display device is shown.

[0171] Step 1302 : When the display device is started, perform collinear feature detection.

[0172] Specifically, when the display device is starting up, an infrared sensor in the eye tracking unit is used to detect a test picture to calculate the coordinates of the center point of the test picture; the test picture has a collinear feature.

[0173] Step 1304: Calculate the actual deformation of the test image and perform image distortion correction.

[0174] Exemplarily, after calculating the actual deformation amount of the test image based on the center point coordinates and the test image with collinear features, image distortion correction is performed; specifically, how to calculate the actual deformation amount of the test image can refer to the content of the above embodiment.

[0175] Step 1306: Acquire geometric features of the user's eye movement for calibration.

[0176] Specifically, the geometric features of the user's eyes are obtained, which serves as the basis for user-customized and accurate gaze point calculation.

[0177] Step 1308: Detect the user's pupil to obtain the user's pupil center coordinates.

[0178] Specifically, an infrared sensor is used to detect the user's pupil features to calculate the current user's pupil center coordinates.

[0179] Step 1310, detecting the user's cornea.

[0180] Specifically, an infrared sensor is used to detect the user's corneal reflection phenomenon and calculate the coordinates of the current user's reflection spot.

[0181] Step 1312: Calculate the user's gaze position.

[0182] Specifically, the user's gaze position is calculated based on the user's pupil coordinates and corneal reflection spot coordinates.

[0183] Step 1314: Render the user's gaze position.

[0184] Specifically, the image is rendered according to the user's current gaze position.

[0185] To better understand the complete process of image correction in the embodiment of the present application, a complete example is used for illustration. Referring to FIG14 , a flow chart of an image correction method in another embodiment is shown, which includes the following steps:

[0186] Step 1402: When the display device is starting up, obtain the currently displayed test image and calculate the current center point coordinates of the test image.

[0187] The test image includes a preset undistorted rectangular area, and the coordinates of the current center point of the test image can be calculated based on the midpoints of the four sides of the preset undistorted rectangular area.

[0188] Step 1404 : Divide the test image into a plurality of concentric areas; set an X-axis and a Y-axis for each area in the divided test image according to the coordinates of the center point of the test image, and set a sampling point for each area.

[0189] Specifically, based on the coordinates of the center point of the test image, a horizontal axis and a vertical axis are set to divide the test image into a plurality of concentric regions according to the coordinates of the center point of the test image, the horizontal axis, and the vertical axis. Sampling points may be set in each region by setting an X-axis and a Y-axis for each concentric region in the test image, using the coordinates of the center point of the test image as an origin, to obtain sampling points for each region.

[0190] Step 1406 : Calculate the first distortion amount and the residual distortion amount after dividing the first distortion amount according to the center point coordinates, and calculate the distortion coefficient corresponding to the residual distortion amount according to the multiple distortion amounts.

[0191] Specifically, the method of calculating the distortion amount and the distortion coefficient can refer to the above steps 904 to 906, which will not be described here.

[0192] Step 1408: When the first type of sampling points are not on the same horizontal line, it is determined that the horizontal distortion coefficient needs to be adjusted.

[0193] The first-class sampling points are the intersections of the x-axis and the boundary of each region. Based on the coordinates of the first-class sampling points, it can be determined whether the first-class sampling points are on the same horizontal line. If the first-class sampling points are not on the same horizontal line, the horizontal distortion coefficient is adjusted.

[0194] In step 1410 , when the second type of sampling points are not located on the same vertical line, it is determined that the vertical distortion coefficient needs to be adjusted.

[0195] The second sampling point is the intersection of the Y-axis set for each region and the boundary of the corresponding region. Based on the coordinate values ​​of the second sampling point, it can be determined whether the second sampling points are on the same vertical line. If the second sampling points are not on the same vertical line, the vertical distortion coefficient is adjusted.

[0196] In step 1412, when the first type of sampling points are on the same horizontal line and the second type of sampling points are on the same vertical line, it is determined that the distortion coefficient does not need to be adjusted.

[0197] Specifically, when the distortion coefficient is adjusted, the distortion coefficient is adjusted to obtain the adjusted distortion coefficient. It is necessary to determine again whether the first-category sampling points are on the same horizontal line and whether the second-category sampling points are on the same vertical line. The distortion coefficient is not adjusted until the first-category sampling points corresponding to the adjusted distortion coefficient are on the same horizontal line and the second-category sampling points are on the same vertical line.

[0198] Step 1414 , select a target sampling point, obtain the original pixel coordinates of the target sampling point; and calculate the distorted pixel coordinates of the target sampling point based on the multiple distortion amounts, the distortion coefficient corresponding to each distortion amount, and the original pixel coordinates.

[0199] Specifically, the method of calculating the distorted pixel coordinates of the target sampling point can refer to steps 208 to 210 and will not be described here.

[0200] Step 1416: Calculate the actual deformation of the test image based on the distorted pixel coordinates and correct the test image.

[0201] Specifically, the method of calculating the actual deformation amount of the test image can refer to steps 212 to 214, which will not be explained here.

[0202] In this embodiment, when a display device is starting up, a currently displayed test image is obtained; the coordinates of the center point of the test image are calculated; based on the center point coordinates, multiple distortion amounts and the distortion coefficient corresponding to each distortion amount are determined; target sampling points are selected and the original pixel coordinates of the target sampling points are obtained; the target sampling points are vertices of each boundary of the test image; the distorted pixel coordinates of the target sampling points are calculated based on the multiple distortion amounts, the distortion coefficient corresponding to each distortion amount, and the original pixel coordinates; the actual deformation amount of the test image is calculated based on the distorted pixel coordinates; and the test image is corrected based on the actual deformation amount. Using this method, when the display device is starting up, the actual deformation amount of the test image can be calculated based on the currently displayed test image, and the test image can be corrected based on the actual deformation amount, thereby achieving customized image distortion optimization for each display device.

[0203] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0204] Based on the same inventive concept, embodiments of the present application also provide an image correction device for implementing the aforementioned image correction method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the image correction device can be found in the above-described limitations of the image correction method and will not be further elaborated here.

[0205] In one embodiment, as shown in FIG15 , an image correction device is provided for use in a display device, including: an image acquisition module 1502, a first coordinate calculation module 1504, a coefficient determination module 1506, a coordinate acquisition module 1508, a second coordinate calculation module 1510, a shape variable calculation module 1512, and a correction module 1514, wherein:

[0206] The image acquisition module 1502 is configured to acquire a currently displayed test image when the display device is starting up;

[0207] A first coordinate calculation module 1504 is used to calculate the coordinates of the center point of the test image;

[0208] A coefficient determination module 1506 is configured to determine a plurality of distortion amounts and a distortion coefficient corresponding to each distortion amount according to the center point coordinates;

[0209] The coordinate acquisition module 1508 is used to select a target sampling point and obtain the original pixel coordinates of the target sampling point; the target sampling point is the vertex of each boundary of the test image;

[0210] A second coordinate calculation module 1510 is configured to calculate the distorted pixel coordinates of the target sampling point based on a plurality of distortion amounts, a distortion coefficient corresponding to each distortion amount, and the original pixel coordinates;

[0211] A shape variable calculation module 1512 is configured to calculate the actual shape variable of the test image based on the distorted pixel coordinates;

[0212] The correction module 1514 is configured to correct the test image according to the actual deformation amount.

[0213] In some embodiments, the test image includes a preset undistorted rectangular area, and the first coordinate calculation module 1504 includes:

[0214] The first coordinate calculation submodule is used to calculate the coordinates of the current center point of the test image according to the midpoints of the four sides of the preset undistorted rectangular area.

[0215] In some embodiments, the coefficient determination module 1506 includes:

[0216] a region division submodule, configured to divide the test image into a plurality of concentric regions based on a horizontal axis, a vertical axis, and the center point coordinates, wherein the center point coordinates of each region are consistent with the center point coordinates of the test image; the plurality of concentric regions include a preset undistorted rectangular region, and the horizontal axis and the vertical axis are set based on the center point coordinates of the test image;

[0217] a distortion amount calculation submodule, configured to calculate, based on the center point coordinates, a first distortion amount and a residual distortion amount obtained by dividing the first distortion amount, wherein a distortion coefficient corresponding to the first distortion amount is a preset first distortion coefficient, and the first distortion amount is a vertical distance from the center point coordinates to a boundary of the preset undistorted rectangular area;

[0218] a first distortion coefficient determination submodule, configured to determine, based on the first distortion amount and a residual distortion amount after dividing the first distortion amount, a distortion coefficient corresponding to the residual distortion amount;

[0219] A coefficient adjustment determination submodule, configured to determine whether the distortion coefficient needs to be adjusted according to preset requirements;

[0220] The first distortion coefficient adjustment submodule is configured to adjust the distortion coefficient when the distortion coefficient needs to be adjusted to obtain the adjusted distortion coefficient.

[0221] In some embodiments, the coefficient adjustment determination submodule includes:

[0222] a sampling point setting unit, configured to set an X-axis and a Y-axis for each region in the divided test image according to the coordinates of the center point of the test image, and to set a sampling point for each region;

[0223] The coefficient adjustment determination unit is used to determine whether the distortion coefficient needs to be adjusted according to the sampling points in each area.

[0224] In some embodiments, the sampling points include first-type sampling points, where the first-type sampling points are intersections of an X-axis set on each region and a boundary of the corresponding region. The coefficient adjustment determination unit includes:

[0225] A first judging subunit is used to judge whether the first type of sampling points in each area are on the same horizontal line;

[0226] The first coefficient determination subunit is configured to determine that the horizontal distortion coefficient needs to be adjusted when the first type of sampling points are not located on the same horizontal line.

[0227] In some embodiments, the sampling points include second-type sampling points, where the second-type sampling points are intersections of a Y-axis set on each region and a boundary of the corresponding region; the distortion coefficient includes a vertical distortion coefficient; and the coefficient adjustment and determination unit further includes:

[0228] The second judging subunit is used to judge whether the second type of sampling points in each area are on the same vertical line;

[0229] The second coefficient determination subunit is configured to determine that the vertical distortion coefficient needs to be adjusted when the second type of sampling points are not located on the same vertical line.

[0230] In some embodiments, the coefficient adjustment determination unit further includes:

[0231] The third coefficient determination subunit is configured to determine that the distortion coefficient does not need to be adjusted when the first-type sampling points are on the same horizontal line and the second-type sampling points are on the same vertical line.

[0232] Each module in the aforementioned image correction device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within the head-mounted display device in hardware form, or may be stored in a memory within the head-mounted display device in software form, allowing the processor to call and execute the corresponding operations of each module.

[0233] In one embodiment, a head-mounted display device is provided. The head-mounted display device may be a terminal, and its internal structure diagram may be shown in Figure 16. The head-mounted display device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the head-mounted display device is used to provide computing and control capabilities. The memory of the head-mounted display device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the head-mounted display device is used to exchange information between the processor and external devices. The communication interface of the head-mounted display device is used to communicate with external terminals via wired or wireless communication. The wireless communication may be achieved via Wi-Fi, a mobile cellular network, NFC (near field communication), or other technologies. When executed by the processor, the computer program implements an image correction method. The display screen of the head-mounted display device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the head-mounted display device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the head-mounted display device shell, or an external keyboard, touchpad or mouse, etc.

[0234] Those skilled in the art will understand that the structure shown in Figure 16 is a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the head-mounted display device to which the solution of the present application is applied. The specific head-mounted display device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0235] In one embodiment, a head-mounted display device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0236] When the display device is starting up, obtaining a currently displayed test image;

[0237] Calculating the center point coordinates of the test image;

[0238] Determining, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts;

[0239] Selecting a target sampling point and obtaining the original pixel coordinates of the target sampling point; the target sampling point is a vertex of each boundary of the test image;

[0240] Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates;

[0241] Calculating the actual deformation of the test image according to the distorted pixel coordinates;

[0242] The test image is corrected according to the actual deformation amount.

[0243] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0244] When the display device is starting up, obtaining a currently displayed test image;

[0245] Calculating the center point coordinates of the test image;

[0246] Determining, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts;

[0247] Selecting a target sampling point and obtaining the original pixel coordinates of the target sampling point; the target sampling point is a vertex of each boundary of the test image;

[0248] Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates;

[0249] Calculating the actual deformation of the test image according to the distorted pixel coordinates;

[0250] The test image is corrected according to the actual deformation amount.

[0251] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0252] When the display device is starting up, obtaining a currently displayed test image;

[0253] Calculating the center point coordinates of the test image;

[0254] Determining, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts;

[0255] Selecting a target sampling point and obtaining the original pixel coordinates of the target sampling point; the target sampling point is a vertex of each boundary of the test image;

[0256] Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates;

[0257] Calculating the actual deformation of the test image according to the distorted pixel coordinates;

[0258] The test image is corrected according to the actual deformation amount.

[0259] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0260] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0261] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0262] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. An image correction method, It is characterized in that Applied to a display device, the method comprises: When the display device is starting up, obtaining a test picture currently displayed; Calculate the center point coordinates of the test image; Determine, according to the center point coordinates, a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts; Select a target sampling point and obtain the original pixel coordinates of the target sampling point; the target sampling point is the vertex of each boundary of the test image; Calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts, and the original pixel coordinates; Calculating the actual deformation amount of the test image according to the distorted pixel coordinates; The test image is corrected according to the actual deformation amount.

2. The method according to claim 1, It is characterized in that The test picture includes a preset undistorted rectangular area, and the calculating the current center point coordinates of the test picture includes: The current center point coordinates of the test image are calculated according to the midpoints of the four sides of the preset undistorted rectangular area.

3. The method according to claim 2, It is characterized in that The step of determining a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts according to the center point coordinates includes: According to the horizontal axis, the vertical axis and the center point coordinates, the test picture is divided into a plurality of concentric areas, wherein the center point coordinates of each area are consistent with the center point coordinates of the test picture; the plurality of concentric areas include a preset undistorted rectangular area, and the horizontal axis and the vertical axis are set based on the center point coordinates of the test picture; According to the center point coordinates, a first distortion amount and a residual distortion amount after dividing the first distortion amount are calculated, wherein a distortion coefficient corresponding to the first distortion amount is a preset first distortion coefficient, and the first distortion amount is a vertical distance from the center point coordinates to a boundary of the preset undistorted rectangular area; Determining a distortion coefficient corresponding to the residual distortion amount according to the first distortion amount and a residual distortion amount after dividing the first distortion amount; Determining whether the distortion coefficient needs to be adjusted according to preset requirements; When the distortion coefficient needs to be adjusted, the distortion coefficient is adjusted to obtain an adjusted distortion coefficient.

4. The method according to claim 3, It is characterized in that The determining whether the distortion coefficient needs to be adjusted according to the preset requirement includes: According to the coordinates of the center point of the test image, an X-axis and a Y-axis are set for each area in the divided test image, and a sampling point is set for each area; According to the sampling points in each area, it is determined whether the distortion coefficient needs to be adjusted.

5. The method according to claim 4, It is characterized in that The sampling points include first-type sampling points, where the first-type sampling points are intersections of an X-axis set on each region and a boundary of the corresponding region; The distortion coefficient includes a horizontal distortion coefficient; and determining whether the distortion coefficient needs to be adjusted according to the sampling points in each area includes: Determine whether the first-class sampling points in each area are on the same horizontal line; When the first type of sampling points are not located on the same horizontal line, it is determined that the horizontal distortion coefficient needs to be adjusted.

6. The method according to claim 5, It is characterized in that The sampling points include second-type sampling points, where the second-type sampling points are intersections of a Y axis set on each region and a boundary of the corresponding region; The distortion coefficient includes a vertical distortion coefficient; and determining whether the distortion coefficient needs to be adjusted according to the sampling points in each area further includes: Determine whether the second type of sampling points in each area are on the same vertical line; When the second-type sampling points are not located on the same vertical line, it is determined that the vertical distortion coefficient needs to be adjusted.

7. The method according to claim 6, It is characterized in that The step of determining whether the distortion coefficient needs to be adjusted according to the sampling points in each area further includes: When the first-type sampling points are on the same horizontal line and the second-type sampling points are on the same vertical line, it is determined that the distortion coefficient does not need to be adjusted.

8. An image correction device, It is characterized in that Applied to a display device, the device comprises: An image acquisition module, used to acquire a currently displayed test image when the display device is starting up; A first coordinate calculation module, used to calculate the coordinates of the center point of the test image; A coefficient determination module, used to determine a plurality of distortion amounts and a distortion coefficient corresponding to each of the distortion amounts according to the coordinates of the center point; A coordinate acquisition module, used to select a target sampling point and acquire the original pixel coordinates of the target sampling point; the target sampling point is the vertex of each boundary of the test image; A second coordinate calculation module, used for calculating the distorted pixel coordinates of the target sampling point according to a plurality of distortion amounts, a distortion coefficient corresponding to each of the distortion amounts and the original pixel coordinates; A shape variable calculation module, used to calculate the actual shape variable of the test image according to the distorted pixel coordinates; A correction module is used to correct the test image according to the actual deformation amount.

9. The device according to claim 8, It is characterized in that The test picture includes a preset undistorted rectangular area, and the first coordinate calculation module includes: The first coordinate calculation submodule is used to calculate the coordinates of the current center point of the test image according to the midpoints of the four sides of the preset undistorted rectangular area.

10. The device according to claim 9, It is characterized in that The coefficient determination module comprises: A region division submodule, configured to divide the test image into a plurality of concentric regions according to a horizontal axis, a vertical axis and the center point coordinates, wherein the center point coordinates of each region are consistent with the center point coordinates of the test image; the plurality of concentric regions include a preset undistorted rectangular region, and the horizontal axis and the vertical axis are set based on the center point coordinates of the test image; a distortion amount calculation submodule, configured to calculate a first distortion amount and a residual distortion amount after dividing the first distortion amount according to the center point coordinates, wherein the distortion coefficient corresponding to the first distortion amount is a preset first distortion coefficient, and the first distortion amount is a vertical distance from the center point coordinates to a boundary of the preset undistorted rectangular area; A first distortion coefficient determination submodule, configured to determine a distortion coefficient corresponding to the residual distortion amount according to the first distortion amount and a residual distortion amount after dividing the first distortion amount; A coefficient adjustment determination submodule, used to determine whether the distortion coefficient needs to be adjusted according to preset requirements; The first distortion coefficient adjustment submodule is used to adjust the distortion coefficient when it is necessary to adjust the distortion coefficient to obtain the adjusted distortion coefficient.

11. The device according to claim 10, It is characterized in that The coefficient adjustment determination submodule includes: A sampling point setting unit, used to set the X-axis and Y-axis for each area in the divided test picture according to the coordinates of the center point of the test picture, and set the sampling point of each area; The coefficient adjustment determination unit is used to determine whether the distortion coefficient needs to be adjusted according to the sampling points in each area.

12. The device according to claim 11, It is characterized in that The sampling points include first-type sampling points, where the first-type sampling points are intersections of an X-axis set on each region and a boundary of a corresponding region. The coefficient adjustment determination unit includes: A first judging subunit is used to judge whether the first type of sampling points in each area are on the same horizontal line; The first coefficient determination subunit is used to determine that the horizontal distortion coefficient needs to be adjusted when the first type of sampling points are not on the same horizontal line.

13. A head mounted display device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the image correction method according to any one of claims 1 to 7 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the image correction method according to any one of claims 1 to 7 are implemented.

15. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the image correction method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Method and device for correcting distortion of image and television set

    CN108305224A

  • Image distortion correction method and device, electronic equipment and readable storage medium

    CN110232667A

  • Image distortion correction method and device

    CN110264426A

  • Image distortion correction method and device

    CN111260567A

  • Imaging distortion correction method and device for AR-HUD system

    CN112381739A