Anti-distortion calibration method and system, electronic device and storage medium

Abstract

The application relates to a reverse distortion calibration method, wherein the method comprises the following steps: drawing a grid chart on a display screen, and obtaining a distortion image by shooting the grid chart through a calibration camera, wherein the calibration camera is arranged at an exit pupil position of a head-mounted device; based on a preset rule, a plurality of original distortion control points are extracted from the distortion image, and the original distortion control points are converted into an optical machine distortion control point in a display coordinate system; a curved surface fitting is performed according to the plurality of optical machine distortion control points, a reverse distortion curved surface Mesh is obtained, a to-be-rendered texture chart is attached to the reverse distortion curved surface Mesh through a shader, and a reverse distortion chart is generated. Through the application, the problems of complex steps and poor efficiency of the reverse distortion calibration method in the related art are solved, and the reverse distortion calibration of an off-axis large-curvature optical system with asymmetrically distributed distortion is realized without manual operation.

Description

Technical Field

[0001] This application relates to the field of augmented reality, and in particular to an anti-distortion calibration method, system, electronic device, and readable storage medium. Background Technology

[0002] In the field of optics, optical distortion caused by optical lens manufacturing processes is a common problem, especially in near-eye real optics, where distortion is one of the important factors affecting user experience.

[0003] In related technologies, current anti-distortion calibration schemes are generally designed for small curvature optical lenses. The distortion distribution of this type of lens is generally centrally symmetrical. Therefore, its distortion parameters can be quickly obtained through finite polynomial fitting, blind guessing, or exhaustive iteration, thereby achieving anti-distortion calibration.

[0004] However, for off-axis optical systems with large curvature, the above methods are not applicable because their optical lenses are not centrally symmetric. For optical systems with asymmetrically distributed distortion, current anti-distortion calibration methods generally suffer from complex procedures and low efficiency, for example:

[0005] 1. Calculate the distorted image of the target image using the physical parameters of the optical lenses, and calculate the phase difference for each pixel to obtain the anti-distortion image;

[0006] The solution has the following drawbacks: calculating the phase difference pixel by pixel is too costly, and due to limitations in manufacturing processes, there is a difference between the theoretical and actual values ​​of lens pixels. Therefore, the anti-distortion image obtained by calculating the phase difference may not be accurate.

[0007] 2. After imaging based on the initial anti-distortion image, the operator checks and confirms the effect in real time, and continuously adjusts the imaging parameters dynamically according to the actual situation to finally obtain an accurate anti-distortion image;

[0008] Although this method can handle both symmetrical and asymmetrical distortions, it requires human intervention and the manual operation steps are relatively complex.

[0009] In summary, no effective solution has yet been proposed to address the issue of poor efficiency in anti-distortion calibration methods in related technologies. Summary of the Invention

[0010] This application provides an anti-distortion calibration method, system, electronic device, and storage medium to at least solve the problem of poor efficiency in related technologies.

[0011] In a first aspect, embodiments of this application provide an anti-distortion calibration method, applied to a head-mounted display device, the method comprising:

[0012] A grid pattern is drawn on the display screen, and a distorted image is obtained by capturing the grid pattern with a calibration camera, wherein the calibration camera is set at the exit pupil position of the head-mounted display device;

[0013] Based on preset rules, multiple original distortion control points are extracted from the distorted image, and the original distortion control points are transformed into the display coordinate system to obtain the optomechanical distortion control points.

[0014] Based on multiple optical-mechanical distortion control points, a surface fitting is performed to obtain an anti-distortion surface mesh. Then, through a shader, the texture map to be rendered is attached to the anti-distortion surface mesh to generate an anti-distortion map.

[0015] In some embodiments, multiple original distortion control points are extracted from the distorted image based on preset rules, including:

[0016] Obtain the imaging center point of the optical engine, and determine the center grid point corresponding to the imaging center point in the distorted image captured by the calibration camera;

[0017] Obtain N adjacent grid points that are adjacent to the central grid point, wherein the adjacent grid points and the central grid point constitute the original distortion control point.

[0018] In some embodiments, transforming the original distortion control points to the display coordinate system includes:

[0019] Calculate the horizontal and vertical intervals of the distorted image based on the central grid point and the adjacent grid points;

[0020] Obtain the grid spacing of the actual grid image displayed on the display, and calculate the conversion ratio of the horizontal and vertical spacing of the distorted image to the grid spacing to obtain scale transformation information;

[0021] Based on the scale transformation information, the original distortion control points are scaled to obtain the optomechanical distortion control points located in the display coordinate system.

[0022] In some embodiments, the horizontal and vertical intervals of the distorted image are calculated based on the central grid point and the adjacent grid points using the following formula;

[0023]

[0024] Where csdx is the horizontal spacing, csdy is the vertical spacing, (u A v A ), (u B v B ), (u C v C), (u D v D ) are the coordinate values ​​of the adjacent grid points.

[0025] In some embodiments, the distortion control points are scaled according to the scale transformation information using the following formula:

[0026]

[0027] Among them, (u a v a ), (u b v b The coordinates of the points in the optical-mechanical distortion control points are (sx, sy), which are the scale transformation information in the horizontal and vertical directions, respectively. csdx is the horizontal interval, and csdy is the vertical spacing.

[0028] In some embodiments, the anti-distortion surface mesh is obtained by surface fitting based on multiple optomechanical distortion control points using the following formula:

[0029]

[0030] Among them, R ij Let (u, v) be the set of coordinates of the optical-mechanical distortion control points, where n and m are the number of grid points in the horizontal and vertical directions, respectively, and (u, v) takes the value range of [0.0, 1.0].

[0031] In some embodiments, after generating the anti-distortion map, the method further includes:

[0032] The anti-distortion image is rendered onto the display screen of the head-mounted display device using a shader, and after optical path cancellation, a distortion-corrected image is generated at the exit pupil position.

[0033] In some embodiments, the field of view of the calibration camera is greater than the field of view of the optical engine of the head-mounted display device, and the optical axis of the calibration camera coincides with the optical axis of the optical engine when capturing the grid pattern.

[0034] In some embodiments, the number of optical-mechanical distortion control points is dynamically adjusted according to the actual application scenario requirements. The more optical-mechanical distortion control points there are, the higher the fitting accuracy of the inverse distortion map.

[0035] Secondly, this application provides an anti-distortion calibration method system applied to a head-mounted display device. The system includes a preprocessing module and a rendering module, wherein...

[0036] The preprocessing module is used to draw a grid pattern on the display screen and obtain a distorted image by capturing the grid pattern with a calibration camera, wherein the calibration camera is positioned at the exit pupil position of the head-mounted display device.

[0037] Furthermore, based on preset rules, multiple original distortion control points are extracted from the distorted image, and the original distortion control points are transformed into the display coordinate system to obtain the optomechanical distortion control points;

[0038] The rendering module is used to perform surface fitting based on multiple optical engine distortion control points to obtain an anti-distortion surface mesh, and to use a shader to attach the texture map to be rendered to the anti-distortion surface mesh to generate an anti-distortion map.

[0039] In some embodiments, the preprocessing module extracts multiple raw distortion control points from the distorted image based on preset rules, including:

[0040] Obtain the imaging center point of the optical engine, and determine the center grid point corresponding to the imaging center point in the distorted image captured by the calibration camera;

[0041] Obtain N adjacent grid points that are adjacent to the central grid point, wherein the adjacent grid points and the central grid point constitute the original distortion control point.

[0042] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0044] Compared to related technologies, the anti-distortion calibration method provided in this application involves drawing a mesh map on the display screen of a head-mounted display device, and capturing the mesh map with a calibration camera positioned at the exit pupil to obtain a distorted image. Further, based on preset rules, multiple original distortion control points are extracted from the distorted image, and these points are transformed into the display coordinate system to obtain optomechanical distortion control points. Finally, surface fitting is performed based on these multiple optomechanical distortion control points to obtain an anti-distortion surface mesh, and a shader is used to attach the texture map to be rendered to the anti-distortion surface mesh to generate an anti-distortion map. This application solves the problems of complex steps and low efficiency in related technologies, enabling anti-distortion calibration of optical systems with large off-axis curvature and asymmetrical distortion distribution without manual intervention. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 This is a schematic diagram of the anti-distortion calibration process based on relevant technologies;

[0047] Figure 2 This is a schematic diagram illustrating the application environment of the anti-distortion calibration method according to an embodiment of this application;

[0048] Figure 3 This is a flowchart of an anti-distortion calibration method according to an embodiment of this application;

[0049] Figure 4 This is a mesh diagram according to an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of a primary distortion control point according to an embodiment of this application;

[0051] Figure 6 It is a mesh image after anti-distortion according to an embodiment of this application;

[0052] Figure 7 This is a structural block diagram of an anti-distortion calibration system according to an embodiment of this application;

[0053] Figure 8 It is an original texture map according to an embodiment of this application;

[0054] Figure 9 Is with Figure 8 The original texture map corresponds to the anti-distortion texture map.

[0055] Figure 10 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0057] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0058] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0059] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0060] In this document, it should be understood that the terms used may be technical means used to implement part of the present invention or other summary technical terms. For example, the terms may include:

[0061] AR (Argument Reality): Augmented Reality is a technology that cleverly integrates virtual information with the real world. It simulates and applies computer-generated virtual information such as text, images, 3D models, and videos to the real world. The two types of information complement each other, thus "enhancing" the real world.

[0062] VR (Virtual Reality): Virtual reality is a virtual world that uses computers and other devices to create a three-dimensional visual experience, giving people in the virtual world a sense of immersion.

[0063] Pose: Position and orientation (facing), for example: in two dimensions it is generally (x, y, yaw), and in three dimensions it is generally (x, y, z, yaw, pitch, roll), including 6 degrees of freedom, 6 of (6 Degrees Of Freedom). The last three elements describe the object's orientation, where yaw is the heading angle, rotating around the Z-axis, pitch is the pitch angle, rotating around the Y-axis, and roll is the roll angle, rotating around the X-axis;

[0064] Mesh: A special type of "mesh" that is a collection of points (also represented by vertices), normal vectors, and faces. The shape of a 3D object can be defined through this mesh.

[0065] Anti-distortion calibration: such as Figure 1 As shown, due to the inherent optical distortion in the manufacturing process of optical lenses, the image seen by the human eye after passing through the lens may not be entirely accurate. In this case, it is necessary to first "convert the image output to the display screen into the corresponding distortion." After this distorted image passes through the lens and reaches the human eye, it will generate a normal image in reverse. This process of "converting the image output to the display screen into the corresponding distortion" is called "anti-distortion calibration."

[0066] The anti-distortion calibration method provided in this application can be applied to, for example, Figure 2 In the scenario shown, Figure 2 This is a schematic diagram illustrating the application environment of the anti-distortion calibration method according to an embodiment of this application, such as... Figure 2 As shown, the head-mounted display device 20 acquires distorted images through a calibration camera positioned at the internal exit pupil, obtains N original distortion control points from these images, converts these original distortion control points into optomechanical distortion control points, and generates an anti-distortion surface mesh based on these control points through surface fitting. Further, the terminal 20 obtains a texture map to be rendered from a local source or from a remote server 21, and after fitting this texture map to the aforementioned anti-distortion surface mesh, generates an anti-distortion image to complete the anti-distortion calibration of the optical system.

[0067] It should be noted that the aforementioned head-mounted display device 20 can be a VR device, AR device, or XR device, and the remote server 21 can be a single server or a cluster of multiple servers.

[0068] Figure 3 This is a flowchart of an anti-distortion calibration method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps:

[0069] S301, draws a grid pattern on the display screen, and obtains a distorted image by capturing the grid pattern through a calibration camera, wherein the calibration camera is set at the exit pupil position of the head-mounted display device;

[0070] The calibration camera can be any common high-definition camera, and its FOV (field of view) should be greater than the FOV of the head-mounted display's own optical engine; and when shooting the grid map, the optical axis of the calibration camera should coincide with the optical axis of the optical engine.

[0071] Figure 4 This is a grid diagram according to an embodiment of this application. The grid diagram drawn on the display screen can be as follows: Figure 4 As shown, the grid diagram has a fixed grid side length, and the features are evenly distributed across the entire display area. It should be noted that in this embodiment, the grid diagram can also be replaced with any pattern with obvious features, as long as it facilitates the acquisition of multiple control points. Specifically, in this embodiment, in addition to the grid diagram, a checkerboard pattern, MarkerGrid, etc., can also be drawn on the display screen for use by the camera at the exit pupil position to capture "distorted images".

[0072] Furthermore, the purpose of setting the calibration camera at the exit pupil position is to simulate the observation position and angle of the human eye and obtain the most accurate anti-distortion image.

[0073] Furthermore, before setting the calibration camera at the exit pupil position, it is necessary to pre-calibrate the intrinsic parameters of the calibration camera and correct the distortion of the calibration camera lens itself. It is understood that the above preprocessing steps are fixed steps that need to be performed and are not related to the core inventive point of this case. Those skilled in the art are also aware of the specific implementation steps; therefore, they will not be described in detail in this embodiment.

[0074] S302, based on preset rules, extracts multiple original distortion control points in the distorted image and transforms the original distortion control points to the display coordinate system to obtain the optomechanical distortion control points;

[0075] After capturing a distorted image using a calibration camera positioned at the exit pupil, multiple discrete original distortion control points are obtained from the distorted image. These distortion control points will then be used to fit a surface that can simulate the inverse distortion distribution.

[0076] Figure 5 This is a schematic diagram of a primary distortion control point according to an embodiment of this application, such as... Figure 5 As shown, the specific process of extracting multiple original distortion control points based on preset rules includes:

[0077] First, determine the imaging center point of the optical engine, and obtain the center grid point O(0,0) corresponding to the imaging center point in the distorted image captured by the calibration camera;

[0078] Secondly, obtain N adjacent grid points that are adjacent to the central grid point. Specifically, for example, select 4 adjacent grid points A(v) A v A ), B(u) B v B ), C(u) c v C ), D(u) D v D All adjacent grid points and the center grid point constitute the original distortion control points.

[0079] It should be noted that the specific number of extracted original distortion control points is not limited in this embodiment and can be dynamically set according to the needs of the actual application scenario. However, it should be understood that the number of original distortion control points is positively correlated with the accuracy of the subsequent fitted surface and negatively correlated with the calibration efficiency. That is, the more original distortion control points there are, the higher the accuracy of the subsequently obtained fitted surface and the higher the accuracy of the anti-distortion calibration, but correspondingly, the processing efficiency will decrease.

[0080] Those skilled in the art can dynamically weigh the accuracy and efficiency aspects according to the actual needs of the scenario, and then select the most suitable number of control points.

[0081] In addition, since the FOV and resolution of the calibration camera and the head-mounted display device's optical engine are different, after obtaining the original distortion control points, it is necessary to perform a scale transformation on the original distortion control points, that is, to transform the original distortion control points to the display coordinate system to obtain the optical engine distortion control points.

[0082] Furthermore, surface fitting is performed based on multiple "optical-mechanical distortion control points" to obtain an anti-distortion surface mesh.

[0083] S303 performs surface fitting based on multiple optical-mechanical distortion control points to obtain an anti-distortion surface mesh, and then uses a shader to attach the texture map to be rendered to the anti-distortion surface mesh to generate an anti-distortion map.

[0084] Specifically, a fitting equation is established based on the bivariate Bernstein polynomial. This equation is used to fit multiple discrete control points into a continuous, anti-distortion distribution, thereby generating a Mesh.

[0085] Finally, based on the obtained Mesh, the texture to be rendered can be attached to the Mesh plane through methods such as OpenGL programming, but not limited to this, to obtain the inverse distortion map corresponding to the texture map.

[0086] Through steps S301 to S303 described above, this embodiment of the application generates a continuous anti-distortion distribution by fitting a surface based on discrete control points. Compared to related technologies that estimate aberrations for each pixel or exhaustively fit distortion parameters, this method is more efficient and the calibration process requires no manual intervention. An anti-distortion map corresponding to any texture map can be generated through a preset automated process. Furthermore, it is compatible with any optical scheme, applicable not only to anti-distortion calibration of optical systems with asymmetric distortion distributions (non-off-axis display optical systems, such as optical waveguides, Birdbath, etc.) but also to optical systems with symmetrical distortion distributions.

[0087] In some embodiments, since the FoV and resolution of the camera module are different from the parameters of the optical engine of the head-mounted display, it is necessary to perform a scale transformation on the coordinate locations extracted from the distorted image captured by the camera at the exit pupil position before fitting the surface. This involves transforming the original coordinate points in the distorted image to the coordinate system of the optical engine display screen. Specifically, this includes:

[0088] First, based on the center grid point and adjacent grid points, the horizontal and vertical intervals of the distorted image are calculated using the following formula;

[0089]

[0090] Where csdx is the horizontal spacing, csdy is the vertical spacing, (u A v A ), (u B v B ), (u C v C ), (u D v D ) are the coordinates of adjacent grid points.

[0091] Furthermore, the grid spacing of the grid image is obtained, and the conversion ratios of the horizontal and vertical spacing of the distorted image with the grid spacing are calculated to obtain scale transformation information. Specifically, the scale transformation relationship between control points in the captured distorted image and control points in the actual image displayed on the monitor is calculated using the following formula:

[0092]

[0093] Among them, sdx and syd are the known grid spacing in the horizontal and vertical directions on the standard grid image displayed on the monitor, respectively, and sx and sy are the scale transformation information in the horizontal and numerical directions, respectively.

[0094] Furthermore, based on this scale transformation information, the coordinates of the control points on the display are calculated using the following formula, which also become the coordinates of the optomechanical distortion control points:

[0095]

[0096] Among them, (u a v a ), (u b v b The specific coordinates of the optical-mechanical distortion control points are (sx, sy), which are the scale transformation ratios in the horizontal and vertical directions, respectively. csdx is the horizontal interval and csdy is the vertical spacing.

[0097] It should be noted that the above formula reflects the calculation method of the coordinates of a specific optomechanical distortion control point (a). It can be understood that the coordinates of other control points are also calculated using the above formula, which will not be elaborated in this embodiment.

[0098] Through the above steps, the scale transformation of multiple control points in the distorted image captured by the calibration camera was completed, resulting in the distorted image on the display, as well as multiple optomechanical distortion control points in the image.

[0099] In some embodiments, after obtaining the optomechanical distortion control points, a surface is fitted based on these discrete optomechanical distortion control points to generate a mesh. Specifically, based on the definition of a bivariate Bernstein polynomial, the fitting is performed using the following formula:

[0100]

[0101] Where Rij is the set of control points mentioned above, n and m are the number of grid points in the horizontal and vertical directions respectively, and the value range of (u, v) is [0.0, 1.0].

[0102] Through the above steps, multiple discrete distortion control points are fitted into an anti-distortion distribution surface, and a mesh is further generated. Subsequently, any texture map to be rendered can be rendered onto the mesh surface to obtain the corresponding anti-distortion map. Figure 6 It is a mesh image after distortion according to an embodiment of this application.

[0103] This embodiment also provides an anti-distortion calibration system for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0104] Figure 7 This is a structural block diagram of an anti-distortion calibration system according to an embodiment of this application, such as... Figure 7 As shown, the system includes: a preprocessing module 70 and a rendering module 71, wherein,

[0105] The preprocessing module 70 is used to draw a grid pattern on the display screen, capture the grid pattern with a calibration camera to obtain a distorted image, wherein the calibration camera is set at the exit pupil position of the head-mounted display device, and extract multiple original distortion control points in the distorted image based on preset rules, and transform the original distortion control points to the display coordinate system to obtain the optomechanical distortion control points.

[0106] Obtain N adjacent grid points that are adjacent to the center grid point, where the adjacent grid points and the center grid point form the original distortion control points; wherein, in the distorted image captured by the calibration camera, the imaging center point of the optical engine is obtained, and the center grid point corresponding to the imaging center point is obtained.

[0107] The rendering module 71 is used to perform surface fitting based on multiple optical engine distortion control points to obtain an anti-distortion surface mesh, and to use a shader to attach the texture map to be rendered to the anti-distortion surface mesh to generate an anti-distortion map. Figure 8 This is an original texture image according to an embodiment of this application. Figure 9 Is with Figure 8 The original texture map corresponds to the anti-distortion texture map.

[0108] It should be noted that optical distortion is an intrinsic parameter of the camera, and distortion correction can be performed offline. Furthermore, after obtaining the distortion correction mapping mesh that matches the optical system, the texture image to be rendered can be obtained locally or online, and then the distortion correction of the texture image to be rendered can be performed on the HMD according to the mesh.

[0109] In one embodiment, Figure 10 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 10 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 10As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores an operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides an environment for the operation of the operating system and computer programs, the computer programs are executed by the processor to implement an anti-distortion calibration method, and the database stores data.

[0110] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0111] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc. .

[0112] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for anti-distortion calibration, characterized in that, When applied to a head-mounted display device, the method includes: A grid pattern is drawn on the display screen, and a distorted image is obtained by capturing the grid pattern with a calibration camera, wherein the calibration camera is set at the exit pupil position of the head-mounted display device; Based on preset rules, multiple original distortion control points are extracted from the distorted image, and the original distortion control points are transformed into the display coordinate system to obtain the optomechanical distortion control points. Based on multiple optical-mechanical distortion control points, a surface fitting is performed to obtain an anti-distortion surface mesh. Then, through a shader, the texture map to be rendered is attached to the anti-distortion surface mesh to generate an anti-distortion map. Multiple original distortion control points are extracted from the distorted image based on preset rules, including: Obtain the imaging center point of the optical engine, and determine the center grid point corresponding to the imaging center point in the distorted image captured by the calibration camera; Obtain N adjacent grid points that are adjacent to the central grid point, wherein the adjacent grid points and the central grid point constitute the original distortion control point.

2. The method according to claim 1, characterized in that, Transforming the original distortion control points to the display coordinate system includes: Calculate the horizontal and vertical intervals of the distorted image based on the central grid point and the adjacent grid points; Obtain the grid spacing of the actual grid image displayed on the display, and calculate the conversion ratio of the horizontal and vertical spacing of the distorted image to the grid spacing to obtain scale transformation information; Based on the scale transformation information, the original distortion control points are scaled to obtain the optomechanical distortion control points located in the display coordinate system.

3. The method according to claim 2, characterized in that, Based on the central grid point and the adjacent grid points, the horizontal and vertical intervals of the distorted image are calculated using the following formula; in, It is the horizontal interval, It is the vertical interval, , , , These are the coordinate values ​​of the adjacent grid points.

4. The method according to claim 3, characterized in that, Based on the scale transformation information, the distortion control points are scaled using the following formula: in, , The coordinates of the point in the optical-mechanical distortion control point, ( ) are the scale transformation information in the horizontal and vertical directions, respectively, where csdx is the horizontal interval and csdy is the vertical interval.

5. The method according to claim 1, characterized in that, Based on multiple optomechanical distortion control points, the anti-distortion surface mesh is obtained by surface fitting using the following formula: in, Let (u, v) be the set of coordinates of the optical-mechanical distortion control points, where n and m are the number of grid points in the horizontal and vertical directions, respectively, and (u, v) takes the value range of [0.0, 1.0].

6. The method according to claim 1, characterized in that, After generating the anti-distortion map, the method further includes: The anti-distortion image is rendered onto the display screen of the head-mounted display device using a shader, and after optical path cancellation, a distortion-corrected image is generated at the exit pupil position.

7. The method according to claim 1, characterized in that, The field of view of the calibration camera is greater than the field of view of the optical engine of the head-mounted display device. When shooting the grid image, the optical axis of the calibration camera coincides with the optical axis of the optical engine.

8. The method according to claim 1, characterized in that, The number of optical-mechanical distortion control points is dynamically adjusted according to the actual application scenario requirements. The more optical-mechanical distortion control points there are, the higher the fitting accuracy of the inverse distortion map.

9. A method system for anti-distortion calibration, characterized in that, When applied to head-mounted display devices, the system includes: a preprocessing module and a rendering module, wherein, The preprocessing module is used to draw a grid pattern on the display screen and obtain a distorted image by capturing the grid pattern with a calibration camera, wherein the calibration camera is positioned at the exit pupil position of the head-mounted display device. Furthermore, based on preset rules, multiple original distortion control points are extracted from the distorted image, and the original distortion control points are transformed into the display coordinate system to obtain the optomechanical distortion control points; The rendering module is used to perform surface fitting based on multiple optical-mechanical distortion control points to obtain an anti-distortion surface mesh, and to use a shader to attach the texture map to be rendered to the anti-distortion surface mesh to generate an anti-distortion map. The preprocessing module extracts multiple original distortion control points from the distorted image based on preset rules, including: Obtain the imaging center point of the optical engine, and determine the center grid point corresponding to the imaging center point in the distorted image captured by the calibration camera; Obtain N adjacent grid points that are adjacent to the central grid point, wherein the adjacent grid points and the central grid point constitute the original distortion control point.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images