Calibration method, device and equipment of projection light machine and storage medium

Abstract

The application relates to the projection technical field and provides a calibration method, device and equipment of a projection light machine and a storage medium, which are applied to a near-eye display device, the near-eye display device comprises a projection light machine and an optical waveguide, the calibration method comprises the following steps: controlling the projection light machine of the near-eye display device to project a preset calibration image on the optical waveguide of the near-eye display device; the preset calibration image comprises a target virtual calibration object; obtaining a target calibration image obtained by a camera shooting the optical waveguide; the target calibration image comprises the preset calibration image projected on the optical waveguide; the camera has a connection relationship with the near-eye display device; determining light machine parameters of the projection light machine according to target calibration images corresponding to a plurality of preset calibration images; the shapes of the target virtual calibration objects in the plurality of preset calibration images are the same, and the rotation angles of the target virtual calibration objects are different, so that the calibration convenience of the projection light machine is improved.

Description

Technical Field

[0001] This application relates to the field of projection technology, and in particular to a calibration method, apparatus, device and storage medium for a projection optical engine. Background Technology

[0002] With the development of near-eye displays, more and more near-eye display devices are incorporating projection engines for near-eye display. Take Augmented Reality (AR) glasses, a type of near-eye display device, as an example. The projection engine on AR glasses can provide a field-of-view effect, displaying virtual images to the user's eyes without obstructing the normal field of vision. Correspondingly, the virtual images can be superimposed and aligned with the actual images of the real environment, thereby achieving an augmented reality effect.

[0003] Due to processing and assembly errors during the actual production and assembly of projection optical engines, discrepancies can easily arise between the actual and theoretical optical engine parameters, necessitating calibration. Therefore, there is an urgent need to improve the ease of calibration for projection optical engines. Summary of the Invention

[0004] The main objective of this application is to provide a calibration method, apparatus, device, and storage medium for a projection optical engine, aiming to solve the technical problem of poor convenience in calibrating projection optical engines.

[0005] In a first aspect, this application provides a calibration method for a projection optical engine, applied to a near-eye display device, the near-eye display device including a projection optical engine and an optical waveguide, the calibration method comprising:

[0006] The projection optical engine of the near-eye display device is controlled to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object.

[0007] A target calibration image is acquired by the camera capturing the optical waveguide; the target calibration image includes a preset calibration image projected onto the optical waveguide; the camera is connected to the near-eye display device.

[0008] The optical engine parameters of the projection optical engine are determined based on the target calibration images corresponding to the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles.

[0009] Secondly, this application provides a calibration device for a projection optical engine, the calibration device comprising:

[0010] An image projection module is used to control the projection optical engine of a near-eye display device to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object;

[0011] An image acquisition module is used to acquire a target calibration image captured by the camera from the optical waveguide, the target calibration image including a virtual target calibration object; the camera is connected to the near-eye display device;

[0012] The projection optical engine calibration module is used to determine the optical engine parameters of the projection optical engine based on the target calibration images corresponding to each of the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles.

[0013] Thirdly, this application provides a near-eye display device, which includes a projection optical engine, an optical waveguide, a memory, and a processor;

[0014] The memory is used to store computer programs;

[0015] The processor is configured to execute the computer program and, in executing the computer program, implement the steps of the calibration method for the projection optical engine as described above.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described calibration method for a projection optical engine.

[0017] This application provides a calibration method, apparatus, device, and storage medium for a projection optical engine. The calibration method includes: controlling the projection optical engine of a near-eye display device to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object; acquiring a target calibration image obtained by a camera capturing images of the optical waveguide; the target calibration image includes the preset calibration image projected onto the optical waveguide; establishing a connection between the camera and the near-eye display device; determining the optical engine parameters of the projection optical engine based on the target calibration images corresponding to each of the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles.

[0018] When multiple preset calibration images contain virtual target calibration objects with identical shapes but different rotation angles, these images can be used to determine the pose relationship between the projection optical engine and virtual target calibration objects at different rotation angles. Correspondingly, since the target calibration images are captured by a camera, and the preset calibration images projected onto the optical waveguide are projections of the projection optical engine, the near-eye display device can determine the pose relationship between the projection optical engine and the camera based on the target calibration images. Therefore, multiple target calibration images can be used to determine the optical engine parameters of the projection optical engine, which improves the ease of calibration. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of a calibration method for a projection optical engine provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the first virtual calibrator involved in the first embodiment of this application;

[0022] Figure 3 This is a schematic block diagram of a calibration device for a projection optical engine provided in an embodiment of this application;

[0023] Figure 4 This is a schematic block diagram of a near-eye display device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] This application provides a calibration method, apparatus, device, and storage medium for a projection optical engine. The calibration method can be applied to near-eye display devices. Near-eye display devices include a projection optical engine and an optical waveguide. No limitations are imposed here. Near-eye display devices may include augmented reality (AR) glasses, mixed reality (MR) glasses, AR helmets, MR helmets, etc., without limitation. The calibration method can also be applied to a server, which can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The near-eye display device can communicate with the server to calibrate the projection optical engine through the server.

[0027] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a calibration method for a projection optical engine provided in an embodiment of this application. It should be noted that the calibration method for the projection optical engine provided in this embodiment can be used in near-eye display devices, which include a projection optical engine and an optical waveguide, and are not limited thereto.

[0029] For example, a near-eye display device includes a projection optical engine and an optical waveguide, both of which are disposed on the near-eye display device. The near-eye display device can control the projection optical engine based on the connection between the near-eye display device and the projection optical engine. Correspondingly, the near-eye display device can also control the optical waveguide based on the connection between the near-eye display device and the optical waveguide. The connection may include at least one of physical connection and communication connection, and is not limited thereto. For instance, when the near-eye display device controls the projection optical engine to project a corresponding image onto the optical waveguide, the optical waveguide can transmit the light corresponding to the image projected onto the optical waveguide to the eye of the user wearing the near-eye display device, allowing the user to view the image projected onto the optical waveguide.

[0030] For example, a near-eye display device can be connected to a camera. This connection can include at least one of a physical connection and a communication connection, without limitation. The camera can be used to simulate the eye movement of a user wearing the near-eye display device to acquire a corresponding image that the user can view, such as a target calibration image. The camera can be a pre-calibrated camera, without limitation. In some embodiments, the camera can include an industrial camera. Through an industrial production line corresponding to the same industrial camera, different near-eye display devices can be moved and positioned at preset calibration locations, thereby calibrating the projection optical engines of different near-eye display devices at the preset calibration locations. However, this is not a limitation and is not intended to be restrictive.

[0031] like Figure 1 As shown, the calibration method for the projection optical engine includes steps S101 to S103.

[0032] S101. Control the projection optical engine of the near-eye display device to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object.

[0033] In some implementations, when the near-eye display device controls the projection optical engine to project a preset calibration image onto the optical waveguide, the preset calibration image can be projected onto the optical waveguide.

[0034] For example, the target virtual calibration object may include calibration points. The number of calibration points may include at least two, and there is no limitation thereto. When the near-eye display device controls the projection optical engine to project a preset calibration image onto the optical waveguide of the near-eye display device, the optical waveguide of the near-eye display device may project the calibration points included in the target virtual calibration object.

[0035] The preset calibration image projected onto the optical waveguide of the near-eye display device can be used to calibrate the optical engine parameters of the projection optical engine.

[0036] For example, a near-eye display device can determine a target virtual benchmark.

[0037] In some implementations, a first virtual calibration object is obtained; the second dimension of the first virtual calibration object is adjusted according to the first dimension of the optical waveguide to obtain a second virtual calibration object; the second virtual calibration object is rotated according to the rotation angle indicated by the preset rotation parameters to obtain target virtual calibration objects corresponding to different rotation angles.

[0038] For example, the first virtual calibration object may include calibration points. The near-eye display device can determine a second size of the first virtual calibration object based on the size of the calibration points and the distance between two adjacent calibration points. Figure 2As shown, taking the calibration points apriltags in the first virtual calibration object as an example. The total number of columns corresponding to the calibration points of the first virtual calibration object is tagClos, and the total number of rows is tagRows. The size of the calibration point includes the side length of the calibration point. Taking the side length of the calibration points as 'a' and the distance between two adjacent calibration points as 'b', the near-eye display device can combine the total number of columns tagClos, the total number of rows tagRows, the side length 'a', and the distance b to determine the second size of the first virtual calibration object.

[0039] The first virtual calibrator and its second dimension can be used to determine the target virtual calibrator.

[0040] When the first dimension of the optical waveguide does not match the second dimension of the first virtual calibration object, if the near-eye display device directly projects the preset calibration image corresponding to the first virtual calibration object onto the optical waveguide of the near-eye display device, it is easy to encounter situations where the preset calibration image cannot be completely projected onto the optical waveguide due to the second dimension being larger than the first dimension, or where blank areas exist on the optical waveguide besides the projected preset calibration image due to the second dimension being smaller than the first dimension. In cases where the preset calibration image cannot be completely projected onto the optical waveguide, or where blank areas exist on the optical waveguide, the positioning accuracy of the calibration points in the preset calibration image is easily affected, thereby affecting the accuracy of subsequent calibration of the optomechanical parameters of the projection optical engine. Therefore, it is necessary to adjust the second dimension of the first virtual calibration object.

[0041] For example, the first dimension of the optical waveguide may include the first width and the first height of the optical waveguide. The second dimension of the first virtual marker may include the second width and the second height of the first virtual marker.

[0042] Taking an optical waveguide with a first width of display_w and a first height of display_h, and a first virtual calibration object with a second width of target_w and a second height of target_h as an example: The near-eye display device can adjust the second width of the first virtual calibration object based on the first width of the optical waveguide. Similarly, the near-eye display device can adjust the second height of the first virtual calibration object based on the first height of the optical waveguide. Correspondingly, the near-eye display device can determine the adjusted first virtual calibration object as the second virtual calibration object.

[0043] The second virtual benchmark can be used to determine the target virtual benchmark.

[0044] For example, during the calibration of the optical engine parameters of a projector, a near-eye display device can calibrate the optical engine parameters using preset calibration images corresponding to the same target virtual calibration object at different rotation angles. Based on this, the near-eye display device can obtain preset rotation parameters when determining the target virtual calibration object. These preset rotation parameters can be used to indicate the rotation angle when rotating the second virtual calibration object. Accordingly, the near-eye display device can rotate the second virtual calibration object according to the rotation angle indicated by the preset rotation parameters to obtain the corresponding target virtual calibration object. Similarly, the near-eye display device can rotate the second virtual calibration object according to different rotation angles indicated by the preset rotation parameters to obtain target virtual calibration objects corresponding to different rotation angles. The preset rotation parameters can be pre-set or user-defined; no restriction is placed here.

[0045] Since the near-eye display device does not change the calibration points included in the first and second virtual calibration objects during the adjustment of the second size of the first virtual calibration object and the rotation of the second virtual calibration object, the target virtual calibration object includes the same calibration points as both the first and second virtual calibration objects. Accordingly, the target virtual calibration object and the first virtual calibration object are equivalent to two calibration objects with the same shape but different scaling ratios. The target virtual calibration object and the second virtual calibration object are equivalent to two calibration objects with the same shape but different rotation angles. Based on this, when the near-eye display device rotates the second virtual calibration object according to different preset rotation parameters, the target virtual calibration objects corresponding to different rotation angles can be identified as multiple calibration objects with the same shape but different rotation angles.

[0046] Thus, when a target virtual calibration object is identified, the near-eye display device can use it to determine the projection parameters of the projection optical engine, which improves the ease of calibration. Similarly, when multiple target virtual calibration objects are identified, the near-eye display device can utilize the fact that these objects have the same shape but different rotation angles to determine the projection parameters of the projection optical engine, further improving both the ease and accuracy of calibration.

[0047] In some implementations, the scaling ratio corresponding to the first virtual calibration object is determined based on the ratio between the first size and the second size; the second size is adjusted according to the scaling ratio to obtain the second virtual calibration object.

[0048] Taking the first width of the optical waveguide as display_w, the first height as display_h, and the second width and second height of the first virtual calibrator as target_w and target_h as an example, a near-eye display device can determine the ratio between the first width display_w and the second width target_w as scale_w. The ratio scale_w can be expressed as... Near-eye display devices can determine the ratio between the first height display_h and the second height display_h as scale_h. The ratio scale_h can be expressed as... The near-eye display device can determine the scaling ratios `scale_w` and `scale_h` as the scaling ratios corresponding to the first virtual calibration object. Accordingly, the near-eye display device can adjust the second width `target_w` based on `scale_w` to obtain the adjusted second width. The near-eye display device can adjust the second height `target_h` based on `scale_h` to obtain the adjusted second height. Based on the adjusted second width and the adjusted second height, the second virtual calibration object can be determined.

[0049] Thus, having obtained the first and second dimensions, the near-eye display device can determine the scaling ratio corresponding to the first virtual calibration object, which improves the ease of determining the scaling ratio of the first virtual calibration object. Correspondingly, this scaling ratio can be used to determine the second virtual calibration object, which improves the ease of determining the second virtual calibration object.

[0050] In some implementations, the second virtual calibration object is rotated according to at least one of the X-axis rotation angle, Y-axis rotation angle, and Z-axis rotation angle indicated by preset rotation parameters to obtain target virtual calibration objects corresponding to different rotation angles.

[0051] When a second virtual calibration object is determined, the near-eye display device can rotate the second virtual calibration object within the space where the projection engine is located, such as performing a 3D rotation operation, to determine the corresponding target virtual calibration object. Preset rotation parameters can be used to indicate at least one of the X-axis rotation angle, Y-axis rotation angle, and Z-axis rotation angle. For example, when the preset rotation parameters indicate an X-axis rotation angle, the near-eye display device can control the second virtual calibration object to rotate by a corresponding rotation angle around the X-axis of the spatial coordinate system where the projection engine is located. Similarly, when the preset rotation parameters indicate a Y-axis rotation angle, the near-eye display device can control the second virtual calibration object to rotate by a corresponding rotation angle around the Y-axis of the spatial coordinate system where the projection engine is located. Likewise, when the preset rotation parameters indicate a Z-axis rotation angle, the near-eye display device can control the second virtual calibration object to rotate by a corresponding rotation angle around the Z-axis of the spatial coordinate system where the projection engine is located. Of course, it's not limited to this. When the preset rotation parameters indicate any two or three of the X-axis rotation angle, Y-axis rotation angle, and Z-axis rotation angle, the near-eye display device can sequentially control the second virtual calibration object to rotate by the corresponding rotation angle according to the indication of any two or three of the X-axis rotation angle, Y-axis rotation angle, and Z-axis rotation angle. The preset rotation parameters can be pre-set or user-defined, and there are no restrictions here.

[0052] For example, a near-eye display device can rotate the same second virtual calibration object according to different preset rotation parameters to obtain target virtual calibration objects corresponding to different rotation angles. These target virtual calibration objects can be multiple calibration objects with the same shape but different rotation angles. The target virtual calibration objects can then be used to calibrate the optical engine parameters of the projection optical engine.

[0053] In this way, the near-eye display device rotates the second virtual calibration object according to the preset rotation parameters to obtain the target virtual calibration object corresponding to different rotation angles, which helps to improve the ease of determining the target virtual calibration object. The target virtual calibration object can be used to calibrate the optical engine parameters of the projection optical engine, which helps to improve the ease of calibration of the projection optical engine.

[0054] S102. Acquire the target calibration image obtained by the camera shooting the optical waveguide; the target calibration image includes a preset calibration image projected onto the optical waveguide; the camera and the near-eye display device are connected.

[0055] For example, when a preset calibration image is projected onto an optical waveguide, the near-eye display device can control the camera to capture a target calibration image of the optical waveguide based on the connection between the near-eye display device and the camera. The target calibration image includes the preset calibration image projected onto the optical waveguide. Correspondingly, if the preset calibration image includes a target virtual calibration object, the target calibration image may also include the target virtual calibration object. If the target virtual calibration object includes calibration points, the target calibration image may also include the corresponding calibration points.

[0056] Thus, when the near-eye display device acquires the target calibration image obtained by the camera capturing the optical waveguide, the target calibration image can be used to calibrate the optomechanical parameters of the projection optical engine.

[0057] S103. Determine the optical engine parameters of the projection optical engine based on the target calibration images corresponding to each of the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles.

[0058] For example, when projecting multiple preset calibration images, since the target virtual calibration object has the same shape in the multiple preset calibration images but different rotation angles, the near-eye display device can change the position information of the target virtual calibration object by changing its rotation angle without changing the pose relationship between the projection optical engine and the camera. Accordingly, the near-eye display device can calibrate the optical engine parameters of the projection optical engine based on the position information of the target virtual calibration object in the corresponding target calibration image at different rotation angles.

[0059] For example, the target virtual calibration object includes calibration points. The near-eye display device can determine the position information of the target virtual calibration object in different target calibration images based on the position information of the calibration points.

[0060] In some implementations, the first position information of each calibration point corresponding to each target calibration image on the target calibration image is determined; the first external parameters of the target virtual calibration object and the camera corresponding to each target calibration image are obtained; based on the first correlation between the first position information and the optical-mechanical parameters, and the second correlation between the first external parameters and the optical-mechanical external parameters, the optical-mechanical parameters of the projection optical engine are determined according to the multiple first position information and the multiple first external parameters.

[0061] For example, when a target calibration image is acquired, the near-eye display device can use an image recognition algorithm to determine the first position information of each calibration point of the target virtual calibration object corresponding to the target calibration image on the target calibration image. The first position information may include 2D coordinates. Taking the j-th calibration point in the target virtual calibration object as an example, the first position information of the j-th calibration point on the target calibration image can be represented as u. ij Similarly, when multiple target calibration images are acquired, the near-eye display device can determine the first position information of each calibration point corresponding to each target calibration image on the target calibration image, thereby determining multiple first position information. Each of the multiple target calibration images corresponds one-to-one with a single first position information.

[0062] For example, when a target virtual calibration object corresponding to a target calibration image is determined, the near-eye display device can calibrate the target virtual calibration object and the first extrinsic parameters of the camera based on the size information of the target virtual calibration object, the first position information of each calibration point of the target virtual calibration object on the target calibration image, and the camera parameters. For instance, the near-eye display device can use a monocular camera pose estimation algorithm, such as the PnP (pespective-n-point) algorithm, combined with the size information of the target virtual calibration object, the first position information, and the camera's intrinsic parameters and camera distortion, to determine the first extrinsic parameters. The camera's intrinsic parameters and camera distortion can be pre-calibrated and are not limited here. Similarly, when multiple target calibration images correspond to target virtual calibration objects with the same shape but different rotation angles, the first extrinsic parameters corresponding to each target calibration image can be determined based on the first position information of each calibration point of the target virtual calibration object on the corresponding target calibration image, thereby determining multiple first extrinsic parameters. Each of the multiple target calibration images corresponds one-to-one with a multiple first extrinsic parameters.

[0063] Since the target calibration image is obtained by capturing a preset calibration image projected onto the optical waveguide by a camera, the virtual target calibration object can be transformed from the spatial coordinate system of the camera to the camera coordinate system using the first extrinsic parameter, and then, under the influence of the camera intrinsic parameter and camera distortion, transformed to the pixel coordinate system of the target calibration image. Since the preset calibration image is projected onto the optical waveguide by a projection optical engine, the preset calibration image is equivalent to transforming the virtual target calibration object from the spatial coordinate system of the projection optical engine to the optical engine coordinate system using the projection optical engine's extrinsic parameter, and then, under the influence of the projection optical engine's intrinsic parameter and optical engine distortion, transformed to the pixel coordinate system of the preset calibration image. When the target calibration image includes the preset calibration image projected onto the optical waveguide, a correlation can be established between the camera parameters and the optical engine parameters. Based on this, the near-eye display device can establish a first correlation between the first position information and the optical engine parameters, and a second correlation between the first extrinsic parameter and the optical engine extrinsic parameter.

[0064] Based on this, given the first position information of each calibration point on the target calibration image and the first external parameter corresponding to each target calibration image, the near-eye display device can determine the optical engine parameters of the projection optical engine by combining the first correlation relationship, the second correlation relationship, multiple first position information and multiple first external parameters.

[0065] For example, optomechanical parameters include intrinsic optomechanical parameters, extrinsic optomechanical parameters, and optomechanical distortion.

[0066] In some implementations, based on the second association relationship, and according to the first extrinsic parameter and the current optical engine extrinsic parameter, the target virtual calibration object corresponding to the target calibration image and the second extrinsic parameter of the projection optical engine are determined; based on the first association relationship, and according to the first position information and the second extrinsic parameter, the current optical engine intrinsic parameter and the current optical engine distortion corresponding to the current optical engine extrinsic parameter are determined; based on a preset nonlinear optimization algorithm, the current optical engine extrinsic parameter, the current optical engine intrinsic parameter and the current optical engine distortion are iteratively optimized to obtain the optical engine extrinsic parameter, the optical engine intrinsic parameter and the optical engine distortion.

[0067] For example, near-eye display devices can substitute the first extrinsic parameter and the current optical engine extrinsic parameter into the second correlation to obtain the second extrinsic parameter of the projection optical engine and the camera.

[0068] The second association includes:

[0069]

[0070] in, Used to indicate the current optical engine extrinsic parameters of the projection optical engine. The first extrinsic parameter used to indicate the relationship between the virtual target calibration object corresponding to the target calibration image and the camera. The second extrinsic parameter used to indicate the target virtual calibration object corresponding to the target calibration image and the projection optical engine.

[0071] Similarly, when the projection optical engine acquires multiple target calibration images, the near-eye display device can combine the first extrinsic parameter corresponding to each target calibration image with the current optical engine extrinsic parameter of the projection optical engine to determine the second extrinsic parameter corresponding to each of the multiple target calibration images.

[0072] The second extrinsic parameter corresponding to each of the multiple target calibration images can be used by near-eye display devices to calibrate the optical engine parameters of the projection optical engine.

[0073] For example, the projection optical engine can substitute the first position information and the second extrinsic parameter into the first correlation to obtain the current optical engine intrinsic parameter and the current optical engine distortion of the projection optical engine.

[0074] The first association includes:

[0075]

[0076] Where n indicates the number of target calibration images, m indicates the number of calibration points in the target virtual calibration object, i indicates the i-th target calibration image, j indicates the j-th calibration point, and K Display D is used to indicate the current optical engine intrinsic parameters of the projection optical engine. Display Used to indicate the current optical engine distortion of the projection optical engine. The second external parameter, u, is used to indicate the virtual target calibration object corresponding to the target calibration image and the projection optical engine. ij X is used to indicate the first position information of the j-th calibration point on the i-th target calibration image. j The second position information of the j-th calibration point in the spatial coordinate system of the projection optical engine is used to indicate the second position information of the j-th calibration point via the current optical engine intrinsic parameter K. Display Second external reference And current camera distortion D Display The third position information of the j-th calibration point in the pixel coordinate system corresponding to the projection optical engine after transformation.

[0077] Near-eye display devices can utilize the first position information u of the j-th calibration point indicated by the target calibration image. ij The difference between the third position information x′ of the j-th calibration point and the second extrinsic parameter corresponding to the target calibration image determined according to the second correlation relationship is used. Determine the current optomechanical intrinsic parameters and current optomechanical distortion corresponding to the current optomechanical extrinsic parameters. When the target virtual calibration object includes multiple calibration points, the near-eye display device can integrate the first and third position information of each calibration point corresponding to the same target calibration image, combined with the second extrinsic parameters corresponding to that target calibration image. The current optomechanical intrinsic parameters and current optomechanical distortion corresponding to the target calibration image are determined. The second extrinsic parameters corresponding to the target calibration image are also determined. Based on the first extrinsic parameter corresponding to the target calibration image and the current optomechanical extrinsic parameter, the near-eye display device can establish the correspondence between the current optomechanical extrinsic parameter, the current optomechanical intrinsic parameter, and the current optomechanical distortion, thereby determining the current optomechanical intrinsic parameter and the current optomechanical distortion as the current optomechanical intrinsic parameter and the current optomechanical distortion corresponding to the current optomechanical extrinsic parameter.

[0078] In some implementations, the second position information of each calibration point corresponding to the target calibration image in the spatial coordinate system of the projection optical engine is obtained; based on the second position information and the second extrinsic parameters, the third position information of each calibration point in the pixel coordinate system corresponding to the projection optical engine and the third association relationship corresponding to the current optical engine parameters are determined; the first position information, the second position information, the third association relationship and the second extrinsic parameters are substituted into the first association relationship to obtain the current optical engine intrinsic parameters and the current optical engine distortion.

[0079] Since the second position information is the position of the virtual target calibration object corresponding to the target calibration image in the spatial coordinates of the projection optical engine, the second position information can include 3D coordinates.

[0080] For example, when a near-eye display device projects a preset calibration image onto an optical waveguide using its projection optical engine, it can adjust the rotation angle of the target virtual calibration object corresponding to the preset calibration image based on the projection information of the preset calibration image on the optical waveguide. For instance, if the projection information indicates that the preset calibration image is not entirely within the corresponding position range of the optical waveguide, and the projection position of the preset calibration image is biased towards the left side of the optical waveguide, the near-eye display device can adjust the rotation angle of the target virtual calibration object to adjust the projection position of the preset calibration image on the optical waveguide to the right. Similarly, if the projection information indicates that the preset calibration image is not entirely within the corresponding position range of the optical waveguide, and the projection position of the preset calibration image is biased towards the upper side of the optical waveguide, the near-eye display device can adjust the rotation angle of the target virtual calibration object to adjust the projection position of the preset calibration image on the optical waveguide downwards. This process continues until the preset calibration image is completely within the corresponding position range of the optical waveguide. Accordingly, when the rotation angle corresponding to the target virtual calibration object is adjusted, the near-eye display device can update the rotation angle corresponding to the target virtual calibration object based on the adjusted rotation angle.

[0081] When the projection information indicates that the preset calibration image is completely within the position range corresponding to the optical waveguide, the near-eye display device can determine the second position information of each calibration point of the target virtual calibration object corresponding to the preset calibration image by combining the position information of the optical waveguide in the spatial coordinate system of the projection optical engine. Accordingly, since the target calibration image includes the preset calibration image projected onto the optical waveguide, the near-eye display device can determine the second position information of each calibration point of the target virtual calibration object corresponding to the preset calibration image as the second position information of each calibration point of the target virtual calibration object corresponding to the target calibration image.

[0082] When the projection information indicates that the preset calibration image is completely within the position range corresponding to the optical waveguide, the near-eye display device can determine the second position information of each calibration point of the target virtual calibration object corresponding to the determined target calibration image as the reference position information of each calibration point.

[0083] Accordingly, since the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles, the near-eye display device can combine the rotation angles of the target virtual calibration objects corresponding to each of the multiple preset calibration images to perform rotation processing on the reference position information of each calibration point, thereby obtaining the second position information of each calibration point corresponding to each of the multiple preset calibration images, that is, determining the second position information of each calibration point corresponding to each of the multiple target calibration images.

[0084] Because the second position information of each calibration point is affected by the optical engine's intrinsic optical parameters, optical engine distortion, and the second extrinsic parameters of the target virtual calibration object and the optical engine when the near-eye display device controls the projection optical engine to project a preset calibration image onto the optical waveguide, the ideal position information of each calibration point in each target calibration image deviates from the first position information. Based on the correlation between the second extrinsic parameter and the first extrinsic parameter and the current optical engine extrinsic parameter of the projection optical engine, the near-eye display device can establish a third correlation relationship between the third position information of each calibration point in the pixel coordinate system corresponding to the projection optical engine and the current optical engine parameters, based on the second position information and the second extrinsic parameter.

[0085] Given the second position information and second extrinsic parameters of each calibration point corresponding to the target calibration image, the near-eye display device can substitute the second position information, the first extrinsic parameters, and the current optomechanical extrinsic parameters into the third correlation relationship to obtain the third correlation relationship between the third position information of the calibration point and the current optomechanical parameters. Correspondingly, given the first position information, the second position information, the third correlation relationship, and the second extrinsic parameters, the first position information, the second position information, the third correlation relationship, and the second extrinsic parameters can be substituted into the first correlation relationship to solve for the current optomechanical intrinsic parameters and the current optomechanical distortion corresponding to the target calibration image, i.e., to determine the current optomechanical intrinsic parameters and the current optomechanical distortion corresponding to the current optomechanical extrinsic parameters.

[0086] Similarly, when multiple target calibration images are acquired, the near-eye display device can determine the current optomechanical intrinsic parameters and current optomechanical distortion corresponding to each of the multiple target calibration images, thereby determining the current optomechanical intrinsic parameters and current optomechanical distortion corresponding to each of the multiple current optomechanical extrinsic parameters, which is equivalent to determining the current optomechanical parameters corresponding to each of the multiple target calibration images.

[0087] For example, given the current optical-mechanical parameters corresponding to multiple target calibration images, the near-eye display device can iteratively optimize the current optical-mechanical extrinsic parameters, current optical-mechanical intrinsic parameters, and current optical-mechanical distortion based on a preset nonlinear optimization algorithm to determine the residual value of the current optical-mechanical parameters corresponding to the target calibration image. Correspondingly, if the residual values ​​corresponding to multiple target calibration images can no longer converge, the near-eye display device can select the current optical-mechanical parameters corresponding to the target calibration image with the smallest residual value from among the multiple target calibration images as the optical-mechanical parameters of the projection optical engine.

[0088] In this way, near-eye display devices can determine the optical engine parameters of the projection optical engine based on the first correlation, the second correlation, multiple first position information and multiple first external parameters, which helps to improve the convenience of calibrating the projection optical engine.

[0089] The calibration method for the projection optical engine provided in the above embodiments includes: controlling the projection optical engine of a near-eye display device to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object; acquiring a target calibration image obtained by a camera capturing the optical waveguide; the target calibration image includes the preset calibration image projected onto the optical waveguide; establishing a connection between the camera and the near-eye display device; determining the optical engine parameters of the projection optical engine based on the target calibration images corresponding to each of the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles.

[0090] When multiple preset calibration images contain virtual target calibration objects with identical shapes but different rotation angles, these images can be used to determine the pose relationship between the projection optical engine and virtual target calibration objects at different rotation angles. Correspondingly, since the target calibration images are captured by a camera, and the preset calibration images projected onto the optical waveguide are projections of the projection optical engine, the near-eye display device can determine the pose relationship between the projection optical engine and the camera based on the target calibration images. Therefore, multiple target calibration images can be used to determine the optical engine parameters of the projection optical engine, which improves the ease of calibration.

[0091] Please see Figure 3 , Figure 3This is a schematic block diagram of a calibration device for a projection optical engine provided in an embodiment of this application. The calibration device can be configured in a near-eye display device or a server to perform the aforementioned calibration method for the projection optical engine. The near-eye display device may include AR glasses, MR glasses, AR helmets, MR helmets, etc., and is not limited thereto. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. The near-eye display device can communicate with the server to calibrate the projection optical engine through the server.

[0092] like Figure 3 As shown, the calibration device for the projection optical engine includes an image projection module 110, an image acquisition module 120, and a projection optical engine calibration module 130.

[0093] The image projection module 110 is used to control the projection optical engine of the near-eye display device to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object.

[0094] The image acquisition module 120 is used to acquire a target calibration image obtained by the camera from the optical waveguide; the target calibration image includes a preset calibration image projected onto the optical waveguide; the camera is connected to the near-eye display device.

[0095] The projection optical engine calibration module 130 is used to determine the optical engine parameters of the projection optical engine based on the target calibration images corresponding to each of the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles.

[0096] For example, the target virtual calibration object includes calibration points. The projection optical engine calibration module 130 includes a first position determination submodule, a first extrinsic parameter determination submodule, and a parameter calibration submodule.

[0097] The first position determination submodule is used to determine the first position information of each calibration point corresponding to each target calibration image on the target calibration image.

[0098] The first extrinsic parameter determination submodule is used to obtain the first extrinsic parameters of the target virtual calibration object and the camera corresponding to each target calibration image.

[0099] The parameter calibration submodule determines the optical-mechanical parameters of the projection optical engine based on the first correlation between the first position information and the optical-mechanical parameters, and the second correlation between the first external parameters and the optical-mechanical external parameters, according to multiple first position information and multiple first external parameters.

[0100] For example, the optomechanical parameters include intrinsic optomechanical parameters, extrinsic optomechanical parameters, and optomechanical distortion. The parameter calibration submodule includes a second extrinsic parameter determination submodule, a first optomechanical parameter determination submodule, and an iterative optimization submodule.

[0101] The second extrinsic parameter determination submodule is used to determine the target virtual calibration object corresponding to the target calibration image and the second extrinsic parameter of the projection optical engine based on the second association relationship, the first extrinsic parameter, and the current optical engine extrinsic parameter.

[0102] The first optomechanical parameter determination submodule is used to determine the current optomechanical intrinsic parameters and the current optomechanical distortion corresponding to the current optomechanical extrinsic parameters based on the first association relationship, the first position information and the second extrinsic parameters.

[0103] The iterative optimization submodule is used to iteratively optimize the current optomechanical extrinsic parameters, the current optomechanical intrinsic parameters, and the current optomechanical distortion based on a preset nonlinear optimization algorithm, so as to obtain the optomechanical extrinsic parameters, the optomechanical intrinsic parameters, and the optomechanical distortion.

[0104] For example, the first optomechanical parameter determination submodule includes a second position determination submodule, a third position determination submodule, and a second optomechanical parameter determination submodule.

[0105] The second position determination submodule is used to obtain the second position information of each calibration point corresponding to the target calibration image in the spatial coordinate system of the projection optical engine.

[0106] The third position determination submodule is used to determine, based on the second position information and the second external parameter, the third position information of each calibration point in the pixel coordinate system corresponding to the projection optical engine and the third association relationship corresponding to the current optical engine parameter.

[0107] The second optomechanical parameter determination submodule is used to substitute the first position information, the second position information, the third association relationship, and the second extrinsic parameter into the first association relationship to obtain the current optomechanical intrinsic parameter and the current optomechanical distortion.

[0108] For example, the calibration device further includes a first calibration object acquisition submodule, a size adjustment submodule, and a first rotation submodule.

[0109] The first calibration object acquisition submodule is used to acquire the first virtual calibration object.

[0110] The size adjustment submodule is used to adjust the second size of the first virtual calibration object according to the first size of the optical waveguide to obtain the second virtual calibration object.

[0111] The first rotation submodule is used to rotate the second virtual calibration object according to the rotation angle indicated by the preset rotation parameters, so as to obtain the target virtual calibration object corresponding to different rotation angles.

[0112] For example, the size adjustment submodule includes a proportion determination submodule and an adjustment submodule.

[0113] The scaling determination submodule is used to determine the scaling ratio corresponding to the first virtual calibration object based on the ratio between the first size and the second size.

[0114] The adjustment submodule is used to adjust the second size according to the scaling ratio to obtain the second virtual calibration object.

[0115] For example, the first rotation submodule includes the second rotation submodule.

[0116] The second rotation submodule is used to rotate the second virtual calibration object according to at least one of the X-axis rotation angle, Y-axis rotation angle, and Z-axis rotation angle indicated by the preset rotation parameters, so as to obtain the target virtual calibration object corresponding to different rotation angles.

[0117] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and its modules and units can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0118] The method of this application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0119] For example, the above-described method and apparatus can be implemented as a computer program that can run on a near-eye display device or a server to calibrate the optical engine parameters of the projection optical engine. For example, the near-eye display device may include AR glasses, MR glasses, AR helmets, MR helmets, etc., without limitation. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0120] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of this application.

[0121] like Figure 4 As shown, the near-eye display device includes a projection optical engine, an optical waveguide, a memory, and a processor. The memory and processor can be connected via a system bus, and the memory may include a storage medium and internal memory.

[0122] The storage medium can store the operating system and computer programs. When the computer program is executed, it enables the processor to perform any calibration method for the projection optical engine.

[0123] The processor provides computing and control capabilities to support the operation of the entire near-eye display device.

[0124] Internal memory provides an environment for the execution of computer programs stored in the storage medium. When these computer programs are executed by the processor, the processor can perform any calibration method for the projector.

[0125] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the near-eye display device to which the present application is applied. A specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0126] It should be understood that a processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other convertible logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0127] In one embodiment, the processor is configured to execute a computer program and, when executing the computer program, perform the following steps:

[0128] The projection optical engine of the near-eye display device is controlled to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object.

[0129] A target calibration image is acquired by the camera capturing the optical waveguide; the target calibration image includes a preset calibration image projected onto the optical waveguide; the camera is connected to the near-eye display device.

[0130] The optical engine parameters of the projection optical engine are determined based on the target calibration images corresponding to the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles.

[0131] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the calibration of the projection optical engine described above can be referred to the corresponding process in the aforementioned embodiment of the calibration method for the projection optical engine, and will not be repeated here.

[0132] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method implemented can be referred to in various embodiments of the calibration method for the projection optical engine of this application.

[0133] The computer-readable storage medium can be an internal storage unit of the near-eye display device described in the foregoing embodiments, such as the hard disk or memory of the near-eye display device. Alternatively, the computer-readable storage medium can be an external storage device of the near-eye display device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the near-eye display device.

[0134] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0135] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0136] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A calibration method for a projection optical engine, characterized in that, The calibration method is applied to near-eye display devices, which include a projection optical engine and an optical waveguide, and includes: The projection optical engine of the near-eye display device is controlled to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object. A target calibration image is acquired by the camera capturing the optical waveguide; the target calibration image includes a preset calibration image projected onto the optical waveguide; the camera is connected to the near-eye display device. The optical-mechanical parameters of the projection optical engine are determined based on the target calibration images corresponding to each of the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles. Before the controlled projection optical engine projects the preset calibration image onto the optical waveguide of the near-eye display device, the calibration method further includes: Obtain the first virtual calibration object; If the first dimension of the optical waveguide does not match the second dimension of the first virtual calibration object, the second dimension of the first virtual calibration object is adjusted according to the first dimension of the optical waveguide to obtain a second virtual calibration object; the first dimension of the optical waveguide includes the first width and the first height of the optical waveguide; the second dimension of the first virtual calibration object includes the second width and the second height of the first virtual calibration object. The second virtual calibration object is rotated according to the rotation angle indicated by the preset rotation parameters to obtain the target virtual calibration object corresponding to different rotation angles; The optomechanical parameters include intrinsic optomechanical parameters, extrinsic optomechanical parameters, and optomechanical distortion. The target virtual calibration object includes calibration points; The step of determining the optical engine parameters of the projection optical engine based on the target calibration image corresponding to each of the plurality of preset calibration images includes: Determine the first position information of each calibration point corresponding to each target calibration image on the target calibration image; Obtain the first extrinsic parameters of the target virtual calibration object and the camera corresponding to each target calibration image; Based on the first correlation between the first position information and the optical engine parameters, and the second correlation between the first external parameters and the optical engine external parameters, the optical engine parameters of the projection optical engine are determined according to multiple first position information and multiple first external parameters. The determination of the optical engine parameters based on the first correlation between the first position information and the optical engine parameters, and the second correlation between the first external parameters and the optical engine external parameters, and according to multiple pieces of the first position information and multiple first external parameters, includes: Based on the second association relationship, and according to the first external parameter and the current optical engine external parameter, the target virtual calibration object corresponding to the target calibration image and the second external parameter of the projection optical engine are determined; Based on the first association relationship, and according to the first location information and the second external parameter, the current optomechanical internal parameter and the current optomechanical distortion corresponding to the current optomechanical external parameter are determined; Based on a preset nonlinear optimization algorithm, the current optomechanical extrinsic parameters, the current optomechanical intrinsic parameters, and the current optomechanical distortion are iteratively optimized to obtain the optomechanical extrinsic parameters, the optomechanical intrinsic parameters, and the optomechanical distortion.

2. The calibration method of claim 1, wherein The step of determining the current optomechanical intrinsic parameters and current optomechanical distortion corresponding to the current optomechanical extrinsic parameters based on the first association relationship, according to the first location information and the second extrinsic parameters, includes: Obtain the second position information of each calibration point corresponding to the target calibration image in the spatial coordinate system of the projection optical engine; Based on the second position information and the second external parameter, determine the third position information of each calibration point in the pixel coordinate system corresponding to the projection optical engine and the third association relationship with the current optical engine parameters; Substituting the first location information, the second location information, the third association relationship, and the second extrinsic parameter into the first association relationship, we obtain the current optomechanical intrinsic parameter and the current optomechanical distortion.

3. The calibration method of claim 1, wherein The step of adjusting the second size of the first virtual calibration object according to the first size of the optical waveguide to obtain the second virtual calibration object includes: Based on the ratio between the first size and the second size, determine the scaling ratio corresponding to the first virtual calibration object; Adjust the second size according to the scaling ratio to obtain the second virtual calibration object.

4. The calibration method of claim 1, wherein The step of rotating the second virtual calibration object according to the rotation angle indicated by the preset rotation parameters to obtain target virtual calibration objects corresponding to different rotation angles includes: The second virtual calibration object is rotated according to at least one of the X-axis rotation angle, Y-axis rotation angle, and Z-axis rotation angle indicated by the preset rotation parameters to obtain target virtual calibration objects corresponding to different rotation angles.

5. A calibration device for a projection optical engine, characterized in that, The calibration device includes: An image projection module is used to control the projection optical engine of a near-eye display device to project a preset calibration image onto the optical waveguide of the near-eye display device; the preset calibration image includes a target virtual calibration object; An image acquisition module is used to acquire a target calibration image captured by a camera on the optical waveguide; the target calibration image includes a preset calibration image projected onto the optical waveguide; the camera is connected to the near-eye display device; The projection optical engine calibration module is used to determine the optical engine parameters of the projection optical engine based on the target calibration images corresponding to each of the multiple preset calibration images; wherein the target virtual calibration objects in the multiple preset calibration images have the same shape and different rotation angles. Before the control projection optical engine projects a preset calibration image onto the optical waveguide of the near-eye display device, the method further includes: Obtain the first virtual calibration object; If the first dimension of the optical waveguide does not match the second dimension of the first virtual calibration object, the second dimension of the first virtual calibration object is adjusted according to the first dimension of the optical waveguide to obtain a second virtual calibration object; the first dimension of the optical waveguide includes the first width and the first height of the optical waveguide; the second dimension of the first virtual calibration object includes the second width and the second height of the first virtual calibration object. The second virtual calibration object is rotated according to the rotation angle indicated by the preset rotation parameters to obtain the target virtual calibration object corresponding to different rotation angles; The optomechanical parameters include intrinsic optomechanical parameters, extrinsic optomechanical parameters, and optomechanical distortion. The target virtual calibration object includes calibration points; The step of determining the optical engine parameters of the projection optical engine based on the target calibration image corresponding to each of the plurality of preset calibration images includes: Determine the first position information of each calibration point corresponding to each target calibration image on the target calibration image; Obtain the first extrinsic parameters of the target virtual calibration object and the camera corresponding to each target calibration image; Based on the first correlation between the first position information and the optical engine parameters, and the second correlation between the first external parameters and the optical engine external parameters, the optical engine parameters of the projection optical engine are determined according to multiple first position information and multiple first external parameters. The determination of the optical engine parameters based on the first correlation between the first position information and the optical engine parameters, and the second correlation between the first external parameters and the optical engine external parameters, and according to multiple pieces of the first position information and multiple first external parameters, includes: Based on the second association relationship, and according to the first external parameter and the current optical engine external parameter, the target virtual calibration object corresponding to the target calibration image and the second external parameter of the projection optical engine are determined; Based on the first association relationship, and according to the first location information and the second external parameter, the current optomechanical internal parameter and the current optomechanical distortion corresponding to the current optomechanical external parameter are determined; Based on a preset nonlinear optimization algorithm, the current optomechanical extrinsic parameters, the current optomechanical intrinsic parameters, and the current optomechanical distortion are iteratively optimized to obtain the optomechanical extrinsic parameters, the optomechanical intrinsic parameters, and the optomechanical distortion.

6. A near-eye display device, characterized in that, The near-eye display device includes a projection optical engine, an optical waveguide, a memory, and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the steps of the calibration method for the projection optical engine as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the calibration method for the projection optical engine as described in any one of claims 1 to 4.

Images