Distortion calibration method, device and equipment of near-eye display device and storage medium

Abstract

The application relates to the technical field of display distortion, and provides a distortion calibration method, device and equipment of a near-eye display device and a storage medium, the method comprising the following steps: controlling a projection module to project a preset calibration image onto a waveguide module; acquiring an image obtained by shooting the waveguide module, wherein the image comprises a target calibration image corresponding to the preset calibration image; determining a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the target calibration image; and determining distortion information of the waveguide module according to the position offset, so as to improve the convenience and comprehensiveness of the distortion calibration of the waveguide module by the near-eye display device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display distortion, and particularly relates to a distortion calibration method and device of a near-eye display device, a near-eye display device and a storage medium. BACKGROUND

[0002] At present, in the process of displaying images by a display of a near-eye display device, due to optical distortion of a display device of the near-eye display device, and due to the influence of optical distortion and distortion caused by device deformation of a waveguide device, catadioptric device and the like of the near-eye display device, the image projected onto a waveguide module by the near-eye display device will be distorted. In the case that the waveguide module of the near-eye display device has an irregular shape, the waveguide module and the contact surface are deformed, the propagation direction of the light changes, thereby causing the generation of barrel distortion, pillow distortion and various types of irregular distortion. However, in the related art, the barrel distortion and the pillow distortion are generally calibrated for subsequent calibration of the barrel distortion and the pillow distortion, and the calibration of the irregular distortion cannot be covered, thereby causing the technical problems of poor convenience of the calibration of the waveguide module and poor comprehensiveness of the calibration of the waveguide module. SUMMARY

[0003] The main purpose of the present application is to provide a distortion calibration method and device of a near-eye display device, and a near-eye display device and a storage medium, aiming at solving the technical problems of poor convenience of the calibration of the waveguide module and poor comprehensiveness of the calibration of the waveguide module caused by the failure to cover the calibration of the irregular distortion.

[0004] In a first aspect, the present application provides a distortion calibration method of a near-eye display device, the near-eye display device comprising a projection module and a waveguide module, and the distortion calibration method comprising:

[0005] controlling the projection module to project a preset calibration image onto the waveguide module;

[0006] obtaining an image obtained by photographing the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image;

[0007] determining a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the target calibration image;

[0008] determining distortion information of the waveguide module according to the position offset.

[0009] In a second aspect, the present application provides a distortion calibration device of a near-eye display device, the distortion calibration device comprising:

[0010] The image projection module is configured to control the projection module to project a preset calibration image onto the waveguide module.

[0011] The image acquisition module is configured to acquire an image obtained by photographing the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image.

[0012] The offset determination module is configured to determine, according to optical flow information between the preset calibration image and the target calibration image, a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image, the position offset comprising a horizontal position offset and a vertical position offset.

[0013] The distortion calibration module is configured to determine, according to the position offset, distortion information of the waveguide module.

[0014] In a third aspect, the present application provides a near-eye display device, comprising a memory and a processor.

[0015] The memory is configured to store a computer program.

[0016] The processor is configured to execute the computer program and implement the steps of the distortion calibration method of the near-eye display device as described above when executing the computer program.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the distortion calibration method of the near-eye display device as described above.

[0018] The present application provides a distortion calibration method, device, equipment and storage medium of a near-eye display device, the distortion calibration method of the near-eye display device comprising: controlling a projection module to project a preset calibration image onto a waveguide module; acquiring an image obtained by photographing the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image; determining, according to optical flow information between the preset calibration image and the target calibration image, a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image; and determining, according to the position offset, distortion information of the waveguide module.

[0019] The near-eye display device can control the projection module to project a preset calibration image on the waveguide module when calibrating the distortion of the waveguide module. In the case that the waveguide module has barrel distortion, pincushion distortion, irregular distortion and the like, when a user observes the preset calibration image projected on the waveguide module, the preset calibration image will have corresponding distortion. Based on this, the near-eye display device can obtain an image obtained by photographing the waveguide module, such as a target calibration image corresponding to the preset calibration image. The near-eye display device can determine the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image. Since each pixel point in the target calibration image corresponds to each pixel point in the waveguide module, the near-eye display device can calibrate the distortion information of the waveguide module according to the position offset, which is conducive to improving the convenience of the near-eye display device in calibrating the distortion of the waveguide module. Moreover, the position offset can be used to determine the distortion information of each pixel point in the waveguide module to cover the barrel distortion, pincushion distortion, irregular distortion and the like that the waveguide module may have, which is conducive to improving the comprehensiveness of the near-eye display device in calibrating the distortion of the waveguide module. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flow diagram of a distortion calibration method of a near-eye display device provided by an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a near-eye display device according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a distortion calibration method of a near-eye display device according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a near-eye display device according to another embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a distortion calibration method of a near-eye display device according to another embodiment of the present application;

[0026] Figure 6 is a schematic block diagram of a distortion calibration device of a near-eye display device provided by an embodiment of the present application;

[0027] Figure 7is a schematic block diagram of a near-eye display device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0029] The flowchart shown in the drawings is only an example and does not necessarily include all the contents and operations / steps, nor does it have to be executed in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0030] The embodiments of the present application provide a distortion calibration method, device and equipment of a near-eye display device and a storage medium. The distortion calibration method of the near-eye display device can be applied to the near-eye display device. The near-eye display device can include an augmented reality (AR) glasses, mixed reality (MR) glasses, AR helmet, MR helmet, etc., without limitation.

[0031] Some embodiments of the present application will be described in detail below with reference to the drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0032] Please refer to Figure 1 , Figure 1 is a flowchart of a distortion calibration method of a near-eye display device provided by an embodiment of the present application. It should be noted that the distortion calibration method of the near-eye display device provided by the embodiments of the present application can be used in a near-eye display device or a server, without limitation.

[0033] As Figure 1 shown, the distortion calibration method of the near-eye display device includes steps S101 to S104.

[0034] S101, control the projection module to project a preset calibration image onto the waveguide module.

[0035] Exemplarily, the near-eye display device comprises a projection module and a waveguide module. The projection module is configured to project an image onto the waveguide module. The waveguide module is configured to display the image projected by the projection module. The number of the projection modules can comprise one or more, and the number of the waveguide modules can comprise one or more, which are not limited herein. The preset calibration image can comprise a checkerboard, a scatter plot, or the like, which is not limited herein.

[0036] Taking the near-eye display device with two projection modules and two waveguide modules as an example. As shown in Figure 2 The projection module can comprise a left projection module and a right projection module. The waveguide module can comprise a left waveguide module and a right waveguide module. The near-eye display device can control the left projection module to project the preset calibration image onto the left waveguide module. The near-eye display device can also control the right projection module to project the preset calibration image onto the right waveguide module, which is not limited herein.

[0037] In the case that the near-eye display device controls the projection module to project the preset calibration image onto the waveguide module, the waveguide module can display the preset calibration image. The near-eye display device can calibrate the distortion of the waveguide module according to the display of the preset calibration image by the waveguide module.

[0038] S102, obtaining an image obtained by photographing the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image.

[0039] For example, the near-eye display device can photograph the waveguide module by means of a photographing module in communication connection with the near-eye display device. The photographing module can comprise a photographing module arranged on the near-eye display device, or an external photographing module, which is not limited herein. The photographing module can be used to simulate the eyes of a user wearing the near-eye display device, and the image obtained by the photographing module photographing the waveguide module can be used to simulate the content viewed by the user when viewing the image projected on the waveguide module. The number of the photographing modules can comprise one or more, which is not limited herein. Taking the near-eye display device in communication connection with one photographing module as an example. As shown in Figure 2As shown, the photographing module can move relative to the near-eye display device. For example, the initial position of the photographing module is set to be close to the area where the left waveguide module of the near-eye display device is located, and then the photographing module can photograph the left waveguide module to obtain a corresponding image. Correspondingly, the position of the photographing module can be changed to be close to the area where the right waveguide module of the near-eye display device is located, and then the photographing module can photograph the right waveguide module to obtain a corresponding image. Of course, the above is not limited, and for another example, the near-eye display device can be connected to two photographing modules. One of the photographing modules is set to be close to the left waveguide module, and then the left waveguide module can be photographed by the one of the photographing modules to obtain a corresponding image. The other of the photographing modules is set to be close to the right waveguide module, and then the right waveguide module can be photographed by the other of the photographing modules to obtain a corresponding image, which is not limited herein.

[0040] When the near-eye display device obtains the image obtained by the photographing module photographing the waveguide module, the near-eye display device can perform image detection on the obtained image to determine whether the proportion between the image area of the waveguide module in the image and the overall image area of the image is greater than or equal to a preset proportion threshold. The near-eye display device can determine that the image is a target calibration image when the proportion between the image area of the waveguide module in the image and the overall image area of the image is greater than or equal to the preset proportion threshold. The near-eye display device can also determine that the image is not a target calibration image when the proportion between the image area of the waveguide module in the image and the overall image area of the image is less than the preset proportion threshold. When the proportion between the image area of the waveguide module in the target calibration image and the overall image area of the target calibration image is greater than or equal to the preset proportion threshold, the target calibration image can better cover the waveguide module to enable the near-eye display device to perform distortion calibration on the waveguide module.

[0041] In some embodiments, since the preset calibration image is projected on the waveguide module, the image obtained by the photographing module photographing the waveguide module can contain the preset calibration image projected on the waveguide module. Correspondingly, the near-eye display device can determine the obtained image obtained by photographing the waveguide module as a target calibration image corresponding to the preset calibration image. The target calibration image can cover the waveguide module and the image content corresponding to the preset calibration image projected on the waveguide module. For example, in the case where the preset calibration image includes random patterns such as a checkerboard and a scatter plot, the target calibration image can include corresponding calibration marks such as a checkerboard and a scatter plot.

[0042] In this way, when the near-eye display device obtains the image obtained by photographing the waveguide module, and the image includes a target calibration image corresponding to the preset calibration image, the near-eye display device can subsequently perform distortion calibration on the waveguide module according to the preset calibration image and the target calibration image.

[0043] S103, determine the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image.

[0044] In some embodiments, when the near-eye display device acquires the preset calibration image and the target calibration image, the near-eye display device can determine the optical flow information between the preset calibration image and the target calibration image. For example, since the preset calibration image and the target calibration image can both include the same calibration mark, the brightness of the same calibration mark does not change when the same calibration mark moves between the preset calibration image and the target calibration image. Based on this, the near-eye display device can track the optical flow information between each calibration mark in the preset calibration image and the corresponding calibration mark in the target calibration image to determine the distortion information of the waveguide module of the near-eye display device. Accordingly, the near-eye display device can establish a position correspondence relationship between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image based on the fact that the preset calibration image and the target calibration image can both include the same calibration mark. Moreover, the near-eye display device can track the optical flow information between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image to determine the distortion information of the waveguide module of the near-eye display device.

[0045] For example, the optical flow information between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image can be represented by a two-dimensional optical flow field. Each vector in the optical flow field represents the position offset of each pixel point in the preset calibration image from the preset calibration image to the target calibration image. The position offset can include a horizontal position offset and a vertical position offset. For example, each vector in the optical flow field can be represented as (u, v). Wherein u is used to indicate the horizontal position offset of the pixel point in the preset calibration image from the preset calibration image to the target calibration image, i.e., the horizontal position offset. v is used to indicate the vertical position offset of the pixel point in the preset calibration image from the preset calibration image to the target calibration image, i.e., the vertical position offset.

[0046] Accordingly, when the waveguide module of the near-eye display device has distortion information, the position information of the corresponding pixel points in the to-be-projected image of the waveguide module and the actual projected image projected onto the waveguide module of the near-eye display device will have a position offset due to the influence of the distortion information of the waveguide module. For example, when the waveguide module has distortion information, there is a position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image.

[0047] Based on this, when the near-eye display device determines the positional offset between each pixel in the preset calibration image and the corresponding pixel in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image, the positional offset can be used by the near-eye display device to determine the distortion information of the waveguide module.

[0048] In some implementations, the target calibration image is subjected to perspective transformation correction processing according to a preset homography transformation function to obtain the processed target calibration image.

[0049] For example, when a preset calibration image and a target calibration image are obtained, the target calibration image, being obtained by photographing the waveguide module, will be affected by perspective transformation compared to the preset calibration image. Therefore, before the near-eye display device determines the positional offset between each pixel in the preset calibration image and the corresponding pixel in the target calibration image, it can first perform perspective transformation correction processing on the target calibration image. This is to avoid the perspective transformation in the target calibration image interfering with the near-eye display device's distortion calibration of the waveguide module, thus affecting the accuracy of the near-eye display device's distortion calibration of the waveguide module.

[0050] like Figure 3 As shown, once the target calibration image is acquired, the near-eye display device can perform perspective transformation correction on the target calibration image according to a preset homography transformation function to obtain a processed target calibration image. The processed target calibration image can then be used by the near-eye display device to perform distortion calibration on the waveguide module and determine its distortion information. Correspondingly, the near-eye display device can also perform distortion correction processing on the processed target calibration image.

[0051] In some implementations, the positional offset between each pixel in the preset calibration image and the corresponding pixel in the processed target calibration image is determined based on the optical flow information between the preset calibration image and the processed target calibration image.

[0052] In a case that the preset calibration image and the processed target calibration image are acquired, the near-eye display device can determine the optical flow information between the preset calibration image and the processed target calibration image. For example, the near-eye display device can track the optical flow information between each calibration mark in the preset calibration image and the corresponding calibration mark in the processed target calibration image to determine the distortion information of the waveguide module of the near-eye display device. Accordingly, the near-eye display device can establish the position correspondence between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image based on that the preset calibration image and the processed target calibration image can both include the same calibration mark. Moreover, the near-eye display device can track the optical flow information between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image to determine the distortion information of the waveguide module of the near-eye display device.

[0053] For example, the near-eye display device can represent the optical flow information between the preset calibration image and the processed target calibration image by an optical flow field. And through the optical flow field, the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image is determined. The position offset can include a horizontal position offset and / or a vertical position offset. The position offset can be used by the near-eye display device to subsequently determine the distortion information of the waveguide module.

[0054] In this way, the near-eye display device determines the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image according to the optical flow information between the preset calibration image and the processed target calibration image, so that the near-eye display device can comprehensively consider the adverse effect of the perspective transformation existing in the target calibration image on the accuracy of the determination of the distortion information of the waveguide module in the process of determining the distortion information of the waveguide module. Based on this, the near-eye display device can avoid the situation that the near-eye display device has poor distortion calibration accuracy of the waveguide module due to the perspective transformation existing in the target calibration image by performing perspective transformation correction processing on the target calibration image and determining the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image in the process of determining the distortion information of the waveguide module, which is conducive to improving the distortion calibration accuracy of the waveguide module of the near-eye display device.

[0055] S104, determining the distortion information of the waveguide module according to the position offset.

[0056] In some embodiments, in a case that the near-eye display device determines the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image, the near-eye display device can determine the distortion information of the waveguide module according to the position offset.

[0057] For example, since the target calibration image is obtained by shooting the waveguide module of the near-eye display device, each pixel point in the target calibration image corresponds to each pixel point in the waveguide module. Based on this, the near-eye display device can determine the position offset of the corresponding pixel point in the waveguide module according to the position offset of each pixel point in the target calibration image. Accordingly, the near-eye display device can determine the distortion information of the waveguide module according to the position offset of each pixel point in the waveguide module.

[0058] Of course, the near-eye display device can determine the position offset of each pixel point in the waveguide module according to the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image, and the one-to-one correspondence between each pixel point in the processed target calibration image and each pixel point in the waveguide module, when determining the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image. The near-eye display device can determine the distortion information of the waveguide module according to the position offset of each pixel point in the waveguide module, which is not limited herein.

[0059] Based on this, when the near-eye display device determines the distortion information of the waveguide module according to the position offset, the distortion information can cover the position offset of each pixel point in the waveguide module. Accordingly, the near-eye display device can cover the distortion information corresponding to the barrel distortion, the pincushion distortion, and the irregular distortion of the waveguide module according to the position offset of each pixel point in the waveguide module, which is beneficial to improve the convenience and comprehensiveness of the near-eye display device in calibrating the distortion of the waveguide module.

[0060] In some embodiments, the near-eye display device can generate a distortion mapping table of the waveguide module of the near-eye display device according to the position offset between the corresponding pixel points in the target calibration image and the preset calibration image. Accordingly, the near-eye display device can store the distortion mapping table of the waveguide module for subsequent distortion correction processing of the to-be-projected image corresponding to the waveguide module.

[0061] For example, when different users use the same near-eye display device, the observation angles of the waveguide module of the near-eye display device by different users can be different because the interpupillary distances of different users can be different. Accordingly, the distortion information of the waveguide module under different observation angles can also change. Based on this, the difference in the distortion information of the near-eye display device when the observation angle is different can be considered comprehensively in the process of calibrating the distortion of the waveguide module of the near-eye display device.

[0062] In some embodiments, a plurality of images obtained by shooting the waveguide module from different directions are obtained, and the plurality of images include a plurality of target calibration images corresponding to the preset calibration image in different directions.

[0063] For example, the shooting module in communication connection with the near-eye display device can shoot the waveguide module from different directions by changing the relative position between the shooting module and the near-eye display device to obtain multiple images including multiple target calibration images corresponding to the preset calibration image in different directions. The near-eye display device can obtain the multiple target calibration images from the multiple images of the waveguide module shot by the shooting module.

[0064] For example, the near-eye display device is in communication connection with a shooting module. As shown in Figure 4 The position of the shooting module can be switched from position 1 to position 2 and from position 2 to position 3. Since the shooting module is used to simulate the eyes of a user wearing the near-eye display device, and the distance between the shooting module in different positions and the plane where the midline of the waveguide module of the near-eye display device is located is also different, it is equivalent that the shooting module in different positions is used to simulate the eyes of a user with different user pupil distances. The observation angles of the eyes of users with different user pupil distances when observing the projection content on the waveguide module of the same near-eye display device are different, and accordingly, the shooting directions of the shooting module when shooting the waveguide module in positions 1, 2 and 3 are different. Based on this, the near-eye display device can obtain multiple images of the waveguide module shot by the external shooting module from different directions based on positions 1, 2 and 3 respectively. In the case that the waveguide module continuously displays a preset calibration image, the shooting module can shoot multiple images covering the preset calibration image from positions 1, 2 and 3 respectively, which is equivalent to obtaining multiple target calibration images corresponding to the preset calibration image in different directions. The user pupil distances corresponding to the multiple target calibration images can be different.

[0065] In some embodiments, the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the multiple target calibration images is determined according to the optical flow information between the preset calibration image and the multiple target calibration images.

[0066] For example, the near-eye display device can perform optical flow tracking on each pixel point in the preset calibration image and the corresponding pixel point in the multiple target calibration images based on the same calibration mark included in the preset calibration image and the multiple target calibration images to determine the optical flow information between the preset calibration image and the multiple target calibration images. Accordingly, the near-eye display device can determine the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the multiple target calibration images according to the optical flow information between the preset calibration image and the multiple target calibration images.

[0067] In some embodiments, the distortion information of the waveguide module is determined according to the multiple position offsets and the user pupil distances corresponding to the multiple target calibration images.

[0068] For example, in a case that the near-eye display device determines the position offset between each pixel in the preset calibration image and the corresponding pixel in each of the plurality of target calibration images, since the user interpupillary distances corresponding to the plurality of target calibration images are different, the near-eye display device can determine the correlation between the position offset and the user interpupillary distance corresponding to each of the plurality of target calibration images, i.e., determine the distortion information of the waveguide module under different user interpupillary distances, thereby determining the distortion information of the waveguide module. For example, the near-eye display device can determine the position offset corresponding to the same target calibration image and the user interpupillary distance based on the target calibration image, and obtain the position offset of the waveguide module under the user interpupillary distance. Accordingly, the near-eye display device can determine the distortion information of the waveguide module under the user interpupillary distance as the distortion information of the waveguide module under the user interpupillary distance. Similarly, the near-eye display device can determine the distortion information of the waveguide module under a plurality of user interpupillary distances, and determine the distortion information of the waveguide module by synthesizing the distortion information under a plurality of user interpupillary distances.

[0069] In this way, the near-eye display device obtains a plurality of images of the waveguide module captured from different directions, and the plurality of images include a plurality of target calibration images corresponding to the preset calibration image in different directions, and the user interpupillary distances corresponding to the plurality of target calibration images are different. The plurality of target calibration images can be used to determine the position offset between each pixel in the preset calibration image and the corresponding pixel in each of the plurality of target calibration images. Moreover, the position offset corresponding to each of the plurality of target calibration images and the user interpupillary distance corresponding to each of the plurality of target calibration images can be used to determine the distortion information of the waveguide module under a plurality of user interpupillary distances, thereby improving the accuracy of the waveguide module distortion calibration of the near-eye display device.

[0070] For example, the near-eye display device can determine the distortion information of the waveguide module under a plurality of user interpupillary distances by using machine learning and deep learning, and determine the distortion information of the waveguide module by synthesizing the distortion information of the waveguide module under a plurality of user interpupillary distances.

[0071] In some embodiments, based on a preset training model, the preset calibration image, a plurality of position offsets, and a plurality of corresponding user interpupillary distances are subjected to distortion recognition training to obtain model optimization parameters; based on the model optimization parameters, the model parameters of the preset training model are adjusted to obtain a distortion recognition model, and the distortion recognition model is used to indicate the distortion information of the waveguide module under a plurality of user interpupillary distances; and based on the distortion recognition model, the distortion information of the waveguide module is determined.

[0072] For example, the near-eye display device can take the preset calibration image as a true value. The near-eye display device can input the preset calibration image as the true value, the plurality of position offsets, and the corresponding plurality of user pupil distances into the preset training model to perform distortion recognition training to obtain model optimization parameters. Based on the model optimization parameters, the model parameters of the preset training model can be adjusted to obtain a distortion recognition model. The distortion recognition model is used to indicate the distortion information of the waveguide module at the plurality of user pupil distances. Based on the determination of the distortion recognition model, the near-eye display device can comprehensively determine the distortion information of the waveguide module based on the distortion information of the waveguide module at the plurality of user pupil distances.

[0073] For example, the near-eye display device can determine the distortion information of the waveguide module at the plurality of user pupil distances by fitting a curve of the plurality of position offsets corresponding to the plurality of target calibration images and the corresponding user pupil distances, to comprehensively determine the distortion information of the waveguide module based on the distortion information of the waveguide module at the plurality of user pupil distances.

[0074] In some embodiments, the user pupil distance includes a first user pupil distance. For example, in a case where the near-eye display device obtains a plurality of target calibration images, the near-eye display device can obtain a first user pupil distance corresponding to each target calibration image.

[0075] According to the position offset and the corresponding first user pupil distance, the distortion information of the waveguide module at the first user pupil distance is determined; the distortion information of the waveguide module at the plurality of first user pupil distances is subjected to curve fitting processing to obtain a distortion recognition curve, the distortion recognition curve being used to indicate the distortion information of the waveguide module at the plurality of first user pupil distances and the distortion information of the waveguide module at at least one second user pupil distance, the second user pupil distance being different from the plurality of first user pupil distances; and the distortion information of the waveguide module is determined according to the distortion recognition curve.

[0076] For example, the near-eye display device can determine the distortion information of the waveguide module at the first user pupil distance according to the position offset corresponding to each target calibration image and the first user pupil distance corresponding to the target calibration image. For example, the position offset corresponding to the target calibration image can be represented as (u, v), and the first user pupil distance corresponding to the target calibration image can be represented as pd. The distortion information of the waveguide module at the first user pupil distance can be represented as (u, v, pd). Accordingly, for a plurality of target calibration images, a plurality of distortion information of the waveguide module at the first user pupil distance corresponding to the target calibration images can be obtained. Since the first user pupil distances corresponding to the plurality of target calibration images can be discrete user pupil distances, the distortion information of the waveguide module at the plurality of first user pupil distances can also be discrete.

[0077] Near-eye display devices can perform curve fitting processing on the distortion information of waveguide modules at multiple first-user interpupillary distances. For example, near-eye display devices can perform B-spline fitting processing on the distortion information of waveguide modules at multiple first-user interpupillary distances. B-spline fitting processing can include least squares fitting, iterative optimization, principal component analysis (PCA) initialization, circular initialization, incremental fitting, offset curve fitting, etc., and is not limited here.

[0078] By performing curve fitting on the distortion information of multiple waveguide modules at multiple first user interpupillary distances, the near-eye display device can obtain a distortion identification curve. The distortion identification curve can be represented, for example, as (u(pd), v(pd)). Since the distortion identification curve is a continuous curve, and the distortion identification model is a curve fitted based on the distortion information of multiple waveguide modules at multiple first user interpupillary distances, the distortion identification curve can also cover the distortion information of waveguide modules at second user interpupillary distances, in addition to the first user interpupillary distances. Based on this, the distortion identification curve is used to indicate the distortion information of multiple waveguide modules at multiple first user interpupillary distances, as well as the distortion information of at least one waveguide module at a second user interpupillary distance, where the second user interpupillary distance is different from all the first user interpupillary distances.

[0079] Accordingly, based on the determination of the distortion recognition model, the near-eye display device can combine the distortion information of the waveguide module at the first user's pupillary distance and the distortion information of the waveguide module at at least one second user's pupillary distance to determine the distortion information of the waveguide module.

[0080] In one exemplary implementation, such as Figure 5 As shown, when a near-eye display device acquires multiple target calibration images of the waveguide module from different directions, it can perform perspective transformation correction on each of these images. For example, by using a homography transformation function, perspective transformation correction can be applied to each of the multiple target calibration images, resulting in processed target calibration images. For these processed target calibration images, the near-eye display device can employ machine learning and deep learning methods to perform distortion calibration on the processed target calibration images and their corresponding user interpupillary distances, obtaining a corresponding distortion recognition model. Alternatively, the near-eye display device can use curve fitting or similar methods to perform distortion calibration on the processed target calibration images and their corresponding user interpupillary distances, obtaining a corresponding distortion recognition curve. Both the distortion recognition model and the distortion recognition curve can be used by the near-eye display device to perform distortion correction on the projected image of the waveguide module.

[0081] In this way, the near-eye display device can determine the distortion information of the waveguide module under multiple user pupil distances in different manners, such as machine learning and deep learning, or curve fitting, to determine the distortion information of the waveguide module under multiple user pupil distances, which facilitates improving the flexibility of the near-eye display device in calibrating the distortion of the waveguide module.

[0082] Correspondingly, the near-eye display device can store the distortion identification model or the distortion identification curve corresponding to the waveguide module, and the near-eye display device can perform distortion correction processing on the to-be-projected image of the waveguide module according to the distortion identification model or the distortion identification curve, which facilitates improving the distortion correction efficiency of the to-be-projected image.

[0083] For example, after determining the distortion information of the waveguide module according to the position offset, the near-eye display device can store the distortion information of the waveguide module for the near-eye display device to perform distortion correction processing on the to-be-projected image of the waveguide module.

[0084] In some embodiments, the to-be-projected image of the waveguide module is obtained, the target projected image corresponding to the to-be-projected image is determined according to the distortion information of the waveguide module, and the projection module is controlled to project the target projected image onto the waveguide module.

[0085] For example, the near-eye display device can obtain the to-be-projected image of the waveguide module for distortion correction processing. The near-eye display device can obtain the distortion information of the waveguide module. The distortion information of the waveguide module can be stored in the storage unit of the near-eye display device, which is not limited herein. After obtaining the distortion information of the waveguide module, the near-eye display device can perform distortion correction processing on the preset positions of the pixel points in the to-be-projected image according to the distortion information of the waveguide module to determine the target projected image corresponding to the to-be-projected image. For example, the near-eye display device can perform distortion correction processing on the to-be-projected image by at least one of the FPGA, the CPU, the GPU, and the like, to obtain the target projected image corresponding to the to-be-projected image, which is not limited herein. After determining the target projected image, the near-eye display device can control the projection module to project the target projected image onto the waveguide module.

[0086] In this way, the near-eye display device can, when the to-be-projected image is obtained, perform distortion correction processing on the to-be-projected image according to the distortion information of the waveguide module to obtain a target projection image. The near-eye display device can control the projection module to project the target projection image onto the waveguide module, which is conducive to improving the convenience of the near-eye display device in performing distortion correction on the to-be-projected image of the waveguide module. By performing distortion correction processing on the to-be-projected image, the user can better observe the target projection image projected on the waveguide module, which is conducive to improving the image projection effect of the near-eye display device on the waveguide module, thereby improving the image display effect of the near-eye display device.

[0087] In some embodiments, a target user interpupillary distance of a user wearing the near-eye display device is obtained; and according to the target user interpupillary distance, distortion information of the waveguide module corresponding to the target user interpupillary distance is determined from at least one of a distortion identification model and a distortion identification curve; and according to the distortion information of the waveguide module corresponding to the target user interpupillary distance, a target projection image corresponding to the to-be-projected image is determined.

[0088] For example, when the near-eye display device obtains the to-be-projected image of the waveguide module to perform distortion correction processing on the to-be-projected image, the near-eye display device can obtain a target user interpupillary distance of a user wearing the near-eye display device. For example, the near-eye display device can capture an eye image of the user wearing the near-eye display device. According to the eye image, the target user interpupillary distance of the user is determined. Of course, this is not limited, and for another example, before capturing the eye image of the user, the near-eye display device can determine whether the user is a new user through voiceprint information, fingerprint information, or other identity verification information of the user. If the user is a new user, the near-eye display device can capture the eye image of the user to determine the target user interpupillary distance of the user. If the user is not a new user, the near-eye display device can obtain the target user interpupillary distance of the user stored in advance according to the identity information of the user. For another example, the target user interpupillary distance of the user can also be input by the user, which is not limited herein.

[0089] When the target user interpupillary distance is obtained, the near-eye display device can input the target user interpupillary distance into at least one of a distortion identification model and a distortion identification curve, so that the near-eye display device can obtain the distortion information of the waveguide module corresponding to the target user interpupillary distance. Accordingly, the near-eye display device can perform distortion correction processing on the to-be-projected image according to the distortion information of the waveguide module corresponding to the target user interpupillary distance to obtain a corresponding target projection image. Of course, this is not limited, for example, the near-eye display device can input the to-be-projected image and the target user interpupillary distance into the distortion identification model, so that the distortion identification model can perform distortion correction processing on the to-be-projected image according to the distortion information of the waveguide module corresponding to the target user interpupillary distance to output a target projection image corresponding to the to-be-projected image, which is not limited herein.

[0090] Thus, the near-eye display device obtains the target user interpupillary distance of the user, and the target user interpupillary distance can be used to determine the distortion information of the waveguide module at the user interpupillary distance. Accordingly, in a case where the distortion information of the waveguide module at the target user interpupillary distance is determined, the near-eye display device can perform distortion correction processing on the to-be-projected image according to the distortion information to obtain a target projected image corresponding to the to-be-projected image, which is conducive to improving the convenience of the near-eye display device in performing distortion correction on the waveguide module.

[0091] The distortion calibration method of the near-eye display device provided in the above embodiment includes: controlling a projection module to project a preset calibration image onto a waveguide module; obtaining an image of the waveguide module, the image including a target calibration image corresponding to the preset calibration image; determining a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the target calibration image; and determining distortion information of the waveguide module according to the position offset.

[0092] When calibrating the distortion of the waveguide module, the near-eye display device can control the projection module to project the preset calibration image onto the waveguide module. In a case where the waveguide module has barrel distortion, pillow distortion, irregular distortion, or the like, when a user observes the projected preset calibration image on the waveguide module, the preset calibration image will have corresponding distortion. Based on this, the near-eye display device can obtain an image of the waveguide module, such as a target calibration image corresponding to the preset calibration image. The near-eye display device can determine a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the target calibration image. Since each pixel point in the target calibration image corresponds to each pixel point in the waveguide module, the near-eye display device can calibrate the distortion information of the waveguide module according to the position offset, which is conducive to improving the convenience of the near-eye display device in calibrating the distortion of the waveguide module. Moreover, the position offset can be used to determine the distortion information of each pixel point in the waveguide module to cover possible barrel distortion, pillow distortion, irregular distortion, or the like of the waveguide module, which is conducive to improving the comprehensiveness of the near-eye display device in calibrating the distortion of the waveguide module.

[0093] Please refer to Figure 6 , Figure 6 is a schematic block diagram of a distortion calibration apparatus of a near-eye display device provided in an embodiment of the present application. The distortion calibration apparatus can be configured in a near-eye display device or a server, and is used to execute the distortion calibration method of the near-eye display device described above. The near-eye display device can include AR glasses, MR glasses, an AR helmet, an MR helmet, or the like, which is not limited herein.

[0094] As Figure 6As shown, the distortion calibration apparatus comprises an image projection module 110, an image acquisition module 120, an offset determination module 130, and a distortion calibration module 140.

[0095] The image projection module 110 is configured to control the projection module to project a preset calibration image onto the waveguide module.

[0096] The image acquisition module 120 is configured to acquire an image obtained by photographing the waveguide module, wherein the image comprises a target calibration image corresponding to the preset calibration image.

[0097] The offset determination module 130 is configured to determine a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the target calibration image.

[0098] The distortion calibration module 140 is configured to determine distortion information of the waveguide module according to the position offset.

[0099] In an exemplary embodiment, the distortion calibration apparatus further comprises a perspective transformation correction sub-module.

[0100] The perspective transformation correction sub-module is configured to perform perspective transformation correction processing on the target calibration image according to a preset homography transformation function to obtain a processed target calibration image.

[0101] The offset determination module 130 comprises a first offset determination sub-module.

[0102] The first offset determination sub-module is configured to determine a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image after perspective correction transformation processing according to optical flow information between the preset calibration image and the processed target calibration image.

[0103] In an exemplary embodiment, the image acquisition module 120 comprises a first image acquisition sub-module.

[0104] The image acquisition sub-module is configured to acquire a plurality of images obtained by photographing the waveguide module from different directions, wherein the plurality of images comprise a plurality of target calibration images corresponding to the preset calibration image in different directions.

[0105] The offset determination module 130 comprises a second offset determination sub-module.

[0106] The second offset determination sub-module is configured to determine a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the plurality of target calibration images.

[0107] The distortion calibration module 140 comprises a first distortion calibration submodule.

[0108] The first distortion calibration submodule is configured to determine distortion information of the waveguide module according to the plurality of position offsets and a plurality of user interpupillary distances respectively corresponding to the plurality of target calibration images.

[0109] In an example, the first distortion calibration submodule comprises a model training submodule, a model parameter adjustment submodule, and a model distortion calibration submodule.

[0110] The model training submodule is configured to perform model optimization parameters on the preset calibration images, the plurality of position offsets, and the corresponding plurality of user interpupillary distances based on a preset training model.

[0111] The model parameter adjustment submodule is configured to adjust model parameters of the preset training model according to the model optimization parameters to obtain a distortion recognition model, the distortion recognition model being configured to indicate the distortion information of the waveguide module under the plurality of user interpupillary distances.

[0112] The model distortion calibration submodule is configured to determine the distortion information of the waveguide module according to the distortion recognition model.

[0113] In an example, the user interpupillary distance comprises a first user interpupillary distance. The first distortion calibration submodule comprises a first distortion information determination submodule, a distortion curve fitting submodule, and a curve distortion calibration submodule.

[0114] The first distortion information determination submodule is configured to determine distortion information of the waveguide module under the first user interpupillary distance according to the position offsets and the corresponding first user interpupillary distance.

[0115] The distortion curve fitting submodule is configured to perform curve fitting processing on the distortion information of the waveguide module under the plurality of first user interpupillary distances to obtain a distortion recognition curve corresponding to the waveguide module, the distortion recognition curve being configured to indicate the distortion information of the waveguide module under the plurality of first user interpupillary distances and the distortion information of the waveguide module under at least one second user interpupillary distance, the second user interpupillary distance being different from the plurality of first user interpupillary distances.

[0116] The curve distortion calibration submodule is configured to determine the distortion information of the waveguide module according to the distortion recognition curve.

[0117] In an example, the distortion calibration apparatus comprises a projection image acquisition submodule, a distortion correction submodule, and an image projection submodule.

[0118] The projection image acquisition submodule is configured to acquire a to-be-projected image of the waveguide module.

[0119] The distortion correction submodule is configured to determine a target projection image corresponding to the to-be-projected image according to distortion information of the waveguide module.

[0120] The image projection submodule is configured to control the projection module to project the target projection image onto the waveguide module.

[0121] For example, the distortion correction submodule includes a target pupil distance acquisition submodule, a second distortion information determination submodule, and a projection image determination submodule.

[0122] The target pupil distance acquisition submodule is configured to acquire a target user pupil distance of a user wearing the near-eye display device.

[0123] The second distortion information determination submodule is configured to determine, from at least one of a distortion identification model and a distortion identification curve, distortion information of the waveguide module at the target user pupil distance according to the target user pupil distance.

[0124] The projection image determination submodule is configured to determine a target projection image corresponding to the to-be-projected image according to the distortion information corresponding to the waveguide module at the target user pupil distance.

[0125] It should be noted that, for the convenience and brevity of description, the specific working processes of the above-described apparatuses and modules and units can be clearly understood by those skilled in the art with reference to the corresponding processes in the foregoing method embodiments, and will not be described here.

[0126] The method of the present application can be used in a variety of general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0127] Exemplarily, the method and the device described above can be implemented in the form of a computer program, which can run on a near-eye display device or a server corresponding to the near-eye display device to calibrate distortion of a waveguide module of the near-eye display device. Exemplarily, the near-eye display device can include AR glasses, MR glasses, an AR helmet, an MR helmet, and the like, which are not limited herein.

[0128] Please refer to Figure 7 , Figure 7 is a structural schematic block diagram of a near-eye display device provided by an embodiment of the present application.

[0129] As Figure 7 shown, the near-eye display device includes a memory and a processor. The memory and the processor can be connected through a system bus. The memory can include a storage medium and an internal memory.

[0130] The storage medium can store an operating system and a computer program. When the computer program is executed, the processor can execute any kind of distortion calibration method of the near-eye display device.

[0131] The processor is configured to provide computing and control capabilities to support the operation of the entire near-eye display device.

[0132] The internal memory provides an environment for the execution of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any kind of distortion calibration method of the near-eye display device.

[0133] Those skilled in the art can understand that Figure 7 the structure shown in the above is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the near-eye display device to which the scheme of the present application is applied. The specific near-eye display device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] It should be understood that the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0135] In one embodiment, the processor is configured to execute the computer program and implement the following steps when executing the computer program:

[0136] controlling the projection module to project a preset calibration image onto the waveguide module;

[0137] obtaining an image of the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image;

[0138] determining a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the target calibration image;

[0139] determining distortion information of the waveguide module according to the position offset.

[0140] It should be noted that, for the convenience and brevity of description, the above description of the specific working process of the distortion calibration of the near-eye display device can refer to the corresponding process in the distortion calibration method of the near-eye display device described above, and will not be repeated here.

[0141] The embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores a computer program. The method implemented by the computer program executed by the processor can refer to each embodiment of the distortion calibration method of the near-eye display device of the present application.

[0142] The computer readable storage medium can be an internal storage unit of the near-eye display device, such as a hard disk or a memory of the near-eye display device. The computer readable storage medium can also be an external storage device of the near-eye display device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0143] It should be understood that the terms used herein in the specification are only for the purpose of describing particular embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, unless otherwise indicated, the singular forms "a", "an", and "the" are intended to include plural forms.

[0144] It should also be understood that, in the specification and the appended claims, the terms "and / or" is used to mean one or more of the associated listed items, as well as any combination of any of the associated listed items. It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

[0145] The above-mentioned embodiment serial numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for distortion calibration of a near-eye display device, the method comprising: The near-eye display device comprises a projection module and a waveguide module, and the distortion calibration method comprises: controlling the projection module to project a preset calibration image onto the waveguide module; the waveguide module comprises a left waveguide module and a right waveguide module; obtaining an image obtained by photographing the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image; determining the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image; determining the distortion information of the waveguide module according to the position offset; the obtaining of the image obtained by photographing the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image, comprises: photographing the waveguide module through a photographing module arranged on the near-eye display device to obtain an image; when the proportion between the image area of the waveguide module in the image and the overall image area of the image is greater than or equal to a preset proportion threshold, the image is determined as a target calibration image; before the determining of the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image, the method further comprises: performing perspective transformation correction processing on the target calibration image according to a preset homography transformation function to obtain a processed target calibration image; the determining of the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image comprises: determining the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the processed target calibration image according to the optical flow information between the preset calibration image and the processed target calibration image.

2. The distortion calibration method of claim 1, wherein, the obtaining of the image obtained by photographing the waveguide module, the image comprising a target calibration image corresponding to the preset calibration image, comprises: obtaining a plurality of images obtained by photographing the waveguide module from different directions, the plurality of images comprising a plurality of target calibration images corresponding to the preset calibration image in different directions; the determining of the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image comprises: determining the position offset between each pixel point in the preset calibration image and the corresponding pixel point in the plurality of target calibration images according to the optical flow information between the preset calibration image and the plurality of target calibration images; the determining of the distortion information of the waveguide module according to the position offset comprises: determining the distortion information of the waveguide module according to a plurality of position offsets and a plurality of user interpupillary distances corresponding to the plurality of target calibration images.

3. The distortion calibration method of claim 2, wherein, the determining of the distortion information of the waveguide module according to a plurality of position offsets and a plurality of user interpupillary distances corresponding to the plurality of target calibration images comprises: based on the preset training model, the preset calibration image, a plurality of position offsets and a plurality of corresponding user interpupillary distances, distortion recognition training is performed to obtain model optimization parameters; According to the model optimization parameters, the model parameters of the preset training model are adjusted to obtain a distortion recognition model, and the distortion recognition model is used to indicate the distortion information of the waveguide module under a plurality of user interpupillary distances. According to the distortion recognition model, the distortion information of the waveguide module is determined.

4. The distortion calibration method of claim 2, wherein, The user interpupillary distance includes a first user interpupillary distance. According to the plurality of position offsets and the user interpupillary distance corresponding to each of the plurality of target calibration images, the distortion information of the waveguide module is determined. According to the position offset and the first user interpupillary distance corresponding thereto, the distortion information of the waveguide module under the first user interpupillary distance is determined. The distortion information of the waveguide module under a plurality of first user interpupillary distances is subjected to curve fitting processing to obtain a distortion recognition curve, and the distortion recognition curve is used to indicate the distortion information of the waveguide module under a plurality of first user interpupillary distances and the distortion information of the waveguide module under at least one second user interpupillary distance, which is different from the plurality of first user interpupillary distances. According to the distortion recognition curve, the distortion information of the waveguide module is determined.

5. The distortion calibration method according to any one of claims 1 to 4, characterized in that, After the distortion calibration method determines the distortion information of the waveguide module according to the position offset, the distortion calibration method further includes: Obtain the to-be-projected image of the waveguide module; According to the distortion information of the waveguide module, the target projection image corresponding to the to-be-projected image is determined; The projection module is controlled to project the target projection image onto the waveguide module.

6. The distortion calibration method of claim 5, wherein, According to the distortion information of the waveguide module, the target projection image corresponding to the to-be-projected image is determined. Obtain the target user interpupillary distance of a user wearing the near-eye display device; According to the target user interpupillary distance, at least one of the distortion recognition model and the distortion recognition curve is used to determine the distortion information of the waveguide module under the target user interpupillary distance; According to the distortion information corresponding to the waveguide module under the target user interpupillary distance, the target projection image corresponding to the to-be-projected image is determined.

7. A distortion calibration apparatus for a near-eye display device, the apparatus comprising: The distortion calibration device includes: An image projection module is configured to control a projection module to project a preset calibration image onto a waveguide module; the waveguide module includes a left waveguide module and a right waveguide module; An image acquisition module is configured to acquire an image obtained by photographing the waveguide module; the image includes a target calibration image corresponding to the preset calibration image; An offset determination module is configured to determine a position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to optical flow information between the preset calibration image and the target calibration image; A distortion calibration module is configured to determine distortion information of the waveguide module according to the position offset; The image acquisition module is configured to acquire an image obtained by photographing the waveguide module; the image includes a target calibration image corresponding to the preset calibration image, which includes: An image acquisition module is configured to acquire an image obtained by photographing the waveguide module; the image includes a target calibration image corresponding to the preset calibration image, which includes: When a proportion of the waveguide module in an image region in the image and a whole image region of the image is greater than or equal to a preset proportion threshold, the image is determined as a target calibration image; Before the step of determining the position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image, the method further comprises: performing perspective transformation correction processing on the target calibration image according to a preset homography transformation function to obtain a processed target calibration image; The step of determining the position offset between each pixel point in the preset calibration image and a corresponding pixel point in the target calibration image according to the optical flow information between the preset calibration image and the target calibration image comprises: determining the position offset between each pixel point in the preset calibration image and a corresponding pixel point in the processed target calibration image according to the optical flow information between the preset calibration image and the processed target calibration image.

8. A near-eye display device, comprising: The near-eye display device comprises a memory and a processor; The memory is configured to store a computer program; The processor is configured to execute the computer program and implement the steps of the distortion calibration method of the near-eye display device according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is configured to be executed by the processor to implement the steps of the distortion calibration method of the near-eye display device according to any one of claims 1 to 6.

Images