Dynamic distortion simulation method and device

Abstract

The invention relates to a dynamic distortion simulation method and device. Obtaining the object plane lattice diagram and optical information corresponding to the near-to-eye display optical system to be simulated; based on the optical information, determining a plurality of first image plane principal ray incident angle lattice diagrams corresponding to the object plane lattice diagrams under a plurality of partial pupil conditions; based on the optical information, determining a second image plane principal ray incident angle lattice diagram corresponding to the object plane lattice diagram under the non-partial pupil condition; determining a dynamic distortion simulation result of the to-be-simulated near-to-eye display optical system based on the plurality of first image plane principal ray incident angle lattice diagrams and the second image plane principal ray incident angle lattice diagrams; according to the method, the adaptability of the to-be-simulated near-to-eye display optical system and the human eyes can be fully considered, the dynamic distortion of the to-be-simulated near-to-eye display optical system is quantified from the angle of the human eyes, and the accuracy of the dynamic distortion simulation result of the near-to-eye display optical system is improved.

Description

Technical Field

[0001] The present application relates to the field of distortion simulation technology, and in particular to a dynamic distortion simulation method and device. Background Art

[0002] Virtual Reality (VR) technology uses computer simulation to generate a digital environment with multiple senses such as vision, hearing, and touch, and enables people to interact with it through auxiliary equipment. It has very wide applications in aerospace, education, medicine, entertainment and other fields. The near-eye display optical system is a key component of virtual display technology, which is responsible for the simulation of visual information.

[0003] The existing near-eye display optical system usually introduces dynamic distortion due to scratches, pitting and other defects on the surface of the optical system, which in turn causes a certain degree of image jitter, jumping and other problems, destroying the authenticity of the virtual environment, causing users to experience dizziness, visual fatigue, and even nausea, seriously affecting the user experience. Because the existing technology cannot accurately quantify the dynamic distortion of the near-eye display optical system, it is impossible to ensure the reliability of the near-eye display optical system, which in turn affects the performance of the near-eye display optical system.

[0004] Currently, no effective solution has been proposed to the problem that the dynamic distortion of near-eye display optical systems cannot be accurately quantified in the prior art. Summary of the invention

[0005] Based on this, it is necessary to provide a dynamic distortion simulation method and device to address the above technical issues.

[0006] In a first aspect, the present application further provides a dynamic distortion simulation method, the method comprising:

[0007] Obtaining an object plane dot pattern and optical information corresponding to the near-eye display optical system to be simulated;

[0008] Based on the optical information, determining a plurality of first image plane chief ray incident angle dot diagrams corresponding to the object plane dot diagram under a plurality of pupil deviation conditions;

[0009] Based on the optical information, determine a second image plane principal ray incident angle dot pattern corresponding to the object plane dot pattern under an unbiased pupil condition;

[0010] Based on the plurality of first image plane principal ray incident angle dot matrix diagrams and the second image plane principal ray incident angle dot matrix diagrams, a dynamic distortion simulation result of the near-eye display optical system to be simulated is determined.

[0011] In one embodiment, obtaining the object plane dot map includes:

[0012] Obtaining a preset image plane principal ray incident angle dot matrix diagram;

[0013] Performing image height processing on the preset image plane chief ray incident angle dot matrix to determine a corresponding first image plane image height dot matrix;

[0014] Ray tracing is performed on the first image plane image height dot map to determine the object plane dot map.

[0015] In one embodiment, the preset image plane principal ray incident angle dot matrix includes a plurality of principal ray incident angle data; the image height processing of the preset image plane principal ray incident angle dot matrix to determine the corresponding first image plane image height dot matrix includes:

[0016] Performing image height processing on the plurality of principal ray incident angle data to obtain a plurality of corresponding image height values;

[0017] According to the plurality of image height values, an image height dot matrix of the first image plane is determined.

[0018] In one embodiment, the optical information includes pupil posture; and determining, based on the optical information, a plurality of first image plane principal ray incident angle dot diagrams corresponding to the object plane dot diagram under a plurality of pupil deviation conditions, comprises:

[0019] Acquire multiple first pupil rotation angles and eye movement function models;

[0020] Based on the eye movement function model and a plurality of the first pupil rotation angles, determining a first pupil deviation amount corresponding to each of the first pupil rotation angles;

[0021] Determining, according to the plurality of first pupil deviations, a pupil position corresponding to each of the first pupil deviations; each of the pupil positions has a corresponding pupil deviation condition;

[0022] Based on the optical information and the object plane dot diagram, a first image plane chief ray incident angle dot diagram corresponding to each pupil deviation condition is determined.

[0023] In one embodiment, before obtaining a plurality of first pupil rotation angles and an eye movement function model, the method includes:

[0024] Obtaining a pupil rotation model and a plurality of second pupil rotation angles;

[0025] Based on the pupil rotation model and the second pupil rotation angles, determining a corresponding plurality of second pupil deviations;

[0026] An eye movement function model is established according to the plurality of second pupil rotation angles and the plurality of second pupil deviation amounts.

[0027] In one embodiment, the optical information includes first optical information and second optical information; and determining the first image plane principal ray incident angle dot map corresponding to each pupil deviation condition based on the optical information and the object plane dot map includes:

[0028] For each of the multiple deviated pupil conditions, based on the first optical information and the object plane dot map, determine a first principal ray incident angle dot map corresponding to each deviated pupil condition; the first optical information refers to optical information corresponding to the near-eye display optical system to be simulated in the absence of target surface defect information;

[0029] Based on the second optical information and the object plane dot pattern, determining a second principal ray incident angle dot pattern corresponding to each pupil deviation condition; the second optical information refers to optical information corresponding to the near-eye display optical system to be simulated in the presence of target surface defect information;

[0030] According to the first principal ray incident angle dot pattern and the second principal ray incident angle dot pattern corresponding to each deviated pupil condition, a first image plane principal ray incident angle dot pattern corresponding to each deviated pupil condition is obtained.

[0031] In one embodiment, before determining the second chief ray incident angle dot pattern corresponding to each pupil deviation condition based on the second optical information and the object plane dot pattern, the method further includes:

[0032] Acquire a preset position and surface defect data; the preset position is located within a preset surface of the near-eye display optical system to be simulated;

[0033] Converting the surface defect data into a certain format to determine target surface defect information;

[0034] The target surface defect information is imported into the preset position to determine the second optical information corresponding to the near-eye display optical system to be simulated.

[0035] In one embodiment, the optical information includes first optical information and second optical information; and determining, based on the optical information, a second image plane principal ray incident angle dot diagram corresponding to the object plane dot diagram under an unbiased pupil condition, comprises:

[0036] Determine a third principal ray incident angle dot diagram corresponding to an unbiased pupil condition based on the first optical information and the object plane dot diagram;

[0037] Determine a fourth principal ray incident angle dot pattern corresponding to an unbiased pupil condition based on the second optical information and the object plane dot pattern;

[0038] A second image plane chief ray incident angle dot matrix diagram is obtained according to the third chief ray incident angle dot matrix diagram and the fourth chief ray incident angle dot matrix diagram.

[0039] In one embodiment, determining the dynamic distortion simulation result of the near-eye display optical system to be simulated based on the plurality of first image plane principal ray incident angle dot patterns and the second image plane principal ray incident angle dot patterns comprises:

[0040] For each of the plurality of deviated pupil conditions, determine a first simulation result corresponding to each deviated pupil condition according to the first principal ray incident angle dot pattern corresponding to each deviated pupil condition and the third principal ray incident angle dot pattern;

[0041] Determine a second simulation result corresponding to each deviated pupil condition according to the second principal ray incident angle dot pattern corresponding to each deviated pupil condition and the fourth principal ray incident angle dot pattern;

[0042] The dynamic distortion simulation result of the near-eye display optical system to be simulated is determined according to the first simulation result and the second simulation result corresponding to each deviated pupil condition.

[0043] In one embodiment, determining the dynamic distortion simulation result of the near-eye display optical system to be simulated according to the first simulation result and the second simulation result corresponding to each pupil deviation condition includes:

[0044] Determine a third simulation result corresponding to each deviated pupil condition according to a difference between the second simulation result corresponding to each deviated pupil condition and the first simulation result;

[0045] According to the third simulation results corresponding to the multiple pupil deviation conditions, the dynamic distortion simulation result of the near-eye display optical system to be simulated is determined.

[0046] In one embodiment, the method further comprises:

[0047] For each of the plurality of deviated pupil conditions, determining a corresponding second image plane image height dot map according to the first principal ray incident angle dot map corresponding to each deviated pupil condition;

[0048] Determine a corresponding third image plane image height dot map according to the second principal ray incident angle dot map corresponding to each pupil deviation condition;

[0049] Generate a text video according to the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition;

[0050] Based on the text video, a dynamic distortion perception model is established.

[0051] In one embodiment, generating a text video according to the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition includes:

[0052] Get the original text image;

[0053] Mapping the second image plane image height dot map and the third image plane image height dot map corresponding to each pupil deviation condition to determine an overall homography matrix corresponding to each pupil deviation condition;

[0054] According to the overall homography matrix corresponding to each skewed pupil condition, mapping processing is performed on the original text image to obtain a mapped text image corresponding to each skewed pupil condition;

[0055] A text video is generated according to the mapped text images corresponding to a plurality of pupil deviation conditions.

[0056] In one embodiment, mapping the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each deviated pupil condition to determine the overall homography matrix corresponding to each deviated pupil condition includes:

[0057] Performing region division on the second image plane image height dot matrix corresponding to each pupil deviation condition to obtain a first number of first divided sub-regions;

[0058] Performing local mapping on a first number of the first divided sub-regions to determine a corresponding plurality of first local homography mapping matrices;

[0059] Performing region division on the third image plane image height dot matrix map corresponding to each pupil deviation condition to obtain a second number of second divided sub-regions; wherein the second number is equal to the first number;

[0060] Performing local mapping on a second number of the second divided sub-regions to determine a corresponding plurality of second local homography mapping matrices;

[0061] According to the multiple first local homography mapping matrices and the multiple second local homography mapping matrices corresponding to each pupil deviation condition, the overall homography matrix corresponding to each pupil deviation condition is determined.

[0062] In one embodiment, determining the overall homography matrix corresponding to each deviated pupil condition according to the plurality of the first local homography mapping matrices and the plurality of the second local homography mapping matrices corresponding to each deviated pupil condition comprises:

[0063] Determining the relative position of each of the first divided sub-regions in the image height dot matrix of the second image plane;

[0064] Determine the relative position of each of the second divided sub-areas in the image height dot matrix of the third image plane;

[0065] Performing spatial distortion removal processing on a first local homography mapping matrix corresponding to the first divided sub-regions and a second local homography mapping matrix corresponding to the second divided sub-regions having the same relative position, to obtain an intermediate local homography mapping matrix corresponding to each relative position;

[0066] According to the plurality of intermediate local homography mapping matrices, a corresponding overall homography matrix is ​​determined.

[0067] In one of the embodiments, the dynamic distortion perception model is sent to a first terminal, so that the first terminal outputs the user's perception result of the dynamic distortion of the near-eye display optical system according to the dynamic distortion perception model.

[0068] In one of the embodiments, an objective function model is established based on the dynamic distortion simulation results of the near-eye display optical system to be simulated and the target surface defect information; the objective function model is used to determine the correlation between dynamic distortion and surface defects;

[0069] The objective function model is sent to the second terminal, so that the second terminal outputs the sensitivity of the near-eye display optical system to surface defects according to the objective function model.

[0070] In a second aspect, the present application further provides a dynamic distortion simulation device, the device comprising:

[0071] An acquisition module, used to acquire an object plane dot pattern and optical information corresponding to a near-eye display optical system to be simulated;

[0072] A first determination module is used to determine, based on the optical information, a plurality of first image plane chief ray incident angle dot diagrams corresponding to the object plane dot diagram under a plurality of deviated pupil conditions;

[0073] A second determination module is used to determine, based on the optical information, a second image plane principal ray incident angle dot pattern corresponding to the object plane dot pattern under an unbiased pupil condition;

[0074] A dynamic distortion simulation module is used to determine the dynamic distortion simulation result of the near-eye display optical system to be simulated based on a plurality of first image plane principal ray incident angle dot matrix diagrams and second image plane principal ray incident angle dot matrix diagrams.

[0075] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in any one of the embodiments of the first aspect are implemented.

[0076] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of the embodiments of the first aspect above are implemented.

[0077] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in any one of the embodiments of the first aspect above.

[0078] The above-mentioned dynamic distortion simulation method and device obtain the optical information corresponding to the object plane dot pattern and the near-eye display optical system to be simulated; based on the optical information, determine the multiple first image plane principal ray incident angle dot patterns corresponding to the object plane dot pattern under multiple deviated pupil conditions, and the second image plane principal ray incident angle dot pattern corresponding to the undeviated pupil condition; because the human eye rotates when observing targets at different positions, which will change the angle at which light enters the eye, therefore based on the first image plane principal ray incident angle dot pattern and the second image plane principal ray incident angle dot pattern, determine the dynamic distortion simulation result of the near-eye display optical system to be simulated, which can fully consider the adaptability of the near-eye display optical system to be simulated and the human eye, realize the quantification of the dynamic distortion of the near-eye display optical system to be simulated from the perspective of the human eye, and improve the accuracy of the dynamic distortion simulation result of the near-eye display optical system. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0080] Figure 1 is an application environment diagram of a dynamic distortion simulation method in an embodiment;

[0081] Figure 2 is a schematic diagram of a flow chart of a dynamic distortion simulation method in one embodiment;

[0082] Figure 3 is a schematic diagram of dynamic distortion simulation in one embodiment;

[0083] Figure 4 A schematic diagram of a process for obtaining an object plane dot map in one embodiment;

[0084] Figure 5 A schematic diagram of a preset image plane chief ray incident angle dot matrix diagram in one embodiment;

[0085] Figure 6is a schematic diagram of an image height dot matrix diagram of a first image plane in one embodiment;

[0086] Figure 7 is a schematic diagram of an object plane dot pattern in one embodiment;

[0087] Figure 8 A schematic diagram of a flow chart of a step of determining a dot matrix diagram of incident angles of a principal ray of a first image plane in one embodiment;

[0088] Fig. 9 is a schematic diagram of the anatomical structure of a human eye in one embodiment;

[0089] Fig.10 is a schematic diagram of a pupil rotation model in one embodiment;

[0090] Fig.11 is a schematic diagram of an eye movement function model in one embodiment;

[0091] Fig.12 A schematic diagram of a flow chart of a step of determining a dot matrix diagram of incident angles of a principal ray of a second image plane in one embodiment;

[0092] Fig.13 A schematic flow chart of the steps of determining the dynamic distortion simulation result of the near-eye display optical system to be simulated in one embodiment;

[0093] Fig.14 is a schematic flow chart of a dynamic distortion simulation method in another embodiment;

[0094] Fig.15 is a schematic diagram of an original text image in one embodiment;

[0095] Fig.16 is a schematic diagram of target surface defect information in one embodiment;

[0096] Fig.17 is a schematic diagram of mapping a text image in one embodiment;

[0097] Fig.18 is a schematic diagram of dividing the second image plane image height dot matrix into regions in one embodiment;

[0098] Fig.19 A schematic diagram of dividing the image height dot matrix of the third image plane into regions in one embodiment;

[0099] Fig. 20 is a schematic flow chart of a dynamic distortion simulation method in a specific embodiment;

[0100] Fig.21 A schematic diagram of a dynamic distortion test device for a near-eye display optical system in one embodiment;

[0101] Fig. 22A schematic diagram of a calibration solution for a dynamic distortion test station in one embodiment;

[0102] Fig.23 A schematic diagram of a cross-lit chart in one embodiment;

[0103] Fig.24 A schematic diagram of a ghost image calibration chart in one embodiment;

[0104] Fig.25 A schematic diagram of a calibration scheme for an internal focusing autocollimator test station in one embodiment;

[0105] Fig.26 is a structural block diagram of a dynamic simulation distortion device in an embodiment;

[0106] Fig. 27 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0107] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0108] The dynamic distortion simulation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, tablets, portable wearable devices, etc.; portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides cloud computing services.

[0109] In an exemplary embodiment, Figure 2 As shown, Figure 2 FIG. 1 is a flow chart of a dynamic distortion simulation method in an embodiment; in this embodiment, the method is applied to Figure 1 The terminal in the example is used for explanation. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.

[0110] In this embodiment, the following steps are included:

[0111] Step S201 , obtaining an object plane dot pattern and optical information corresponding to a near-eye display optical system to be simulated.

[0112] The object plane dot map includes the intersection position of each group of light rays on the object plane; the object plane refers to the plane where the virtual image source is located or the position image source of the actual light source, such as a display screen. The near-eye display optical system to be simulated may be, but is not limited to, virtual reality (VR) and augmented reality (AR) devices, which are not specifically limited here. The optical information corresponding to the near-eye display optical system to be simulated may include, but is not limited to, geometric parameters, material properties, surface shape, etc.

[0113] Step S202, based on the optical information, determining a plurality of first image plane chief ray incident angle dot diagrams corresponding to a plurality of pupil deviation conditions of the object plane dot diagram.

[0114] The first image plane principal ray incident angle dot pattern refers to the incident angle distribution of the principal ray when the object plane dot pattern reaches the image plane through the near-eye display optical system to be simulated under the condition of pupil deviation. The pupil deviation condition refers to the situation where the aperture in the optical system deviates from the optical axis, for example, when the eyeball rotates to a non-0° viewing angle. The incident angle of the principal ray is the angle between the principal ray and the optical axis.

[0115] For example, see Figure 3 , Figure 3 H2 is a schematic diagram of the object plane; T1 is a schematic diagram of the near-eye display optical system to be simulated; T2 is a schematic diagram of the aperture, which is used to simulate the pupil of the human eye; H2 is a schematic diagram of the image plane; it should be noted that by changing the position of the aperture T2, different deviated pupil conditions can be met. The object plane dot pattern is used as the image source of the object plane H2. Based on the position of the aperture T2, the corresponding deviated pupil condition can be determined, and then based on the optical information of the near-eye display optical system T1 to be simulated, the first image plane principal ray incident angle dot pattern on the image plane H2 under the current deviated pupil condition can be determined by ray tracing. Similarly, by changing the position of the aperture T2, multiple first image plane principal ray incident angle dot patterns corresponding to multiple deviated pupil conditions can be determined. It should be noted that ray tracing is used to simulate the propagation path of light in the scene to calculate the interaction between light and the surface of the object.

[0116] Step S203, based on the optical information, determining the second image plane principal ray incident angle dot pattern corresponding to the object plane dot pattern under the condition of unbiased pupil.

[0117] The unbiased pupil condition refers to the situation where the aperture in the optical system is located on the optical axis, for example, when the eyeball rotates to a viewing angle of 0°.

[0118] For example, Figure 3 Taking the schematic diagram in as an example, when the aperture T2 is located on the optical axis of the system, it can be determined that this is the unbiased pupil condition. The object plane dot pattern is used as the image source of the object plane H2. Based on the optical information of the near-eye display optical system T1 to be simulated, the second image plane principal light incident angle dot pattern on the image plane H2 under the unbiased pupil condition can be determined by ray tracing.

[0119] Step S204, determining a dynamic distortion simulation result of the near-eye display optical system to be simulated based on a plurality of first image plane chief ray incident angle dot patterns and a second image plane chief ray incident angle dot pattern.

[0120] The dynamic distortion simulation results include the position, shape and degree of the dynamic distortion; wherein the degree of distortion is represented by the peak value of the change in the incident angle of the main light.

[0121] It should be noted that, by determining the dot matrix of the principal ray incident angle of the first image plane corresponding to multiple deviated pupil conditions and the dot matrix of the principal ray incident angle of the second image plane corresponding to the undeviated pupil condition, it is possible to accurately determine the change in the incident angle of the principal ray on a specific image plane under different deviated pupil conditions, i.e., different field angles, and then based on the peak value of the change in the incident angle of the principal ray, the dynamic distortion degree of the near-eye display optical system to be simulated can be accurately quantified to obtain the corresponding dynamic distortion simulation result.

[0122] Exemplarily, an object plane dot diagram and optical information corresponding to the near-eye display optical system to be simulated are obtained; based on the optical information, a first image plane principal ray incident angle dot diagram corresponding to a plurality of deviated pupil conditions and a second image plane principal ray incident angle dot diagram corresponding to an undeviated pupil condition are determined; for each deviated pupil condition, in the first image plane principal ray incident angle dot diagram corresponding to the deviated pupil condition, a change in the principal ray incident angle corresponding to the current deviated pupil condition is determined based on the first image plane principal ray incident angle dot diagram corresponding to the current deviated pupil condition and the second image plane principal ray incident angle dot diagram corresponding to the undeviated pupil condition, and so on, a change in the principal ray incident angle corresponding to a plurality of deviated pupil conditions is determined; and then, based on the change in the principal ray incident angle corresponding to the plurality of deviated pupil conditions, a dynamic distortion simulation result of the near-eye display optical system to be simulated is determined.

[0123] It is understandable that in order to fully consider the compatibility of the near-eye display optical system to be simulated with the human eye, the quantification of the degree of dynamic distortion is achieved based on the change in the incident angle of the main light ray. This can not only accurately predict the performance of the system under different field of view angles, but also provide important reference indicators for optimizing the near-eye display optical system to be simulated and improving the user experience.

[0124] In this embodiment, based on optical information, multiple first image plane principal ray incident angle dot patterns corresponding to multiple deviated pupil conditions and the second image plane principal ray incident angle dot pattern corresponding to the undeviated pupil condition can be accurately determined; based on the first image plane principal ray incident angle dot pattern and the second image plane principal ray incident angle dot pattern, the dynamic distortion simulation result of the near-eye display optical system to be simulated is determined, which can fully consider the adaptability of the near-eye display optical system to be simulated and the human eye, realize the quantification of the dynamic distortion of the near-eye display optical system to be simulated from the perspective of the human eye, and improve the accuracy of the dynamic distortion simulation result of the near-eye display optical system.

[0125] In one embodiment, Figure 4 As shown, Figure 4 The figure is a flow chart of the steps of obtaining an object plane dot map in one embodiment; obtaining an object plane dot map includes the following steps:

[0126] Step S401, obtaining a preset image plane principal ray incident angle dot matrix.

[0127] The preset image plane principal ray incident angle dot matrix diagram includes a plurality of principal ray incident angle data; the principal ray incident angle data refers to the angle data between the principal ray and the optical axis.

[0128] It should be noted that the preset image plane principal ray incident angle dot matrix needs to be set according to actual simulation requirements and is not specifically limited here. Figure 5 , the field of view of the preset image plane chief ray incident angle dot pattern can be set to meet ±40° and the dot spacing to meet 1°.

[0129] It should be noted that the method of obtaining the preset image plane principal ray incident angle dot pattern may be, but is not limited to, ray tracing.

[0130] Step S402, performing image height processing on a preset image plane chief ray incident angle dot matrix to determine a corresponding first image plane image height dot matrix.

[0131] The first image plane image height dot matrix includes multiple image height values; the image height value refers to the distance from the image point to the center of the image plane. It should be noted that for each incident angle of the main ray, the corresponding image height can be calculated through the geometric relationship of the optical system, and then the corresponding first image plane image height dot matrix can be obtained according to the multiple image heights.

[0132] The method of performing image height processing on the preset image plane principal ray incident angle dot map may include, but is not limited to, using a tangent function to determine the corresponding first image plane image height dot map.

[0133] Preferably, performing image height processing on a preset image plane principal ray incident angle dot matrix to determine a corresponding first image plane image height dot matrix comprises the following steps:

[0134] Step 1: Perform image height processing on multiple principal ray incident angle data to obtain multiple corresponding image height values.

[0135] Step 2: determining the first image plane image height dot map according to multiple image height values.

[0136] For example, Figure 3 The schematic diagram and Figure 5 As an example, let's take the preset image plane principal ray incident angle dot diagram in as an example. If the distance from the aperture T2 to the image plane H2 is T, Figure 5 The image height is processed based on the preset image plane principal ray incident angle dot matrix in the image plane, and a plurality of corresponding image height values ​​are obtained. The specific process is as follows: based on the principal ray incident angle β in the preset image plane principal ray incident angle dot matrix, and the distance T from the aperture T2 to the image plane H2, the image height value H corresponding to each principal ray incident angle β can be determined based on formula (1). Further, based on the plurality of image height values ​​H, the first image plane image height dot matrix can be determined. The first image plane image height dot matrix is ​​as follows: Figure 6 shown.

[0137] H=T×tan(β) (1)

[0138] Among them, H represents the image height value, the unit is mm; T represents the distance from the aperture to the image plane, the unit is mm; β represents the incident angle of the main light, the unit is deg.

[0139] Step S403, performing ray tracing on the first image plane image height dot map to determine the object plane dot map.

[0140] For example, based on Figure 6 The first image plane image height dot matrix diagram in the image plane and the optical information corresponding to the near-eye display optical system to be simulated are used to perform reverse ray tracing on the first image plane image height dot matrix diagram, that is, tracing from the image plane to the object plane in reverse order to determine the corresponding object plane dot matrix diagram. The object plane dot matrix diagram is as follows: Figure 7 shown.

[0141] In this embodiment, based on the preset image plane principal ray incident angle dot diagram, the corresponding first image plane image height dot diagram can be accurately calculated, and then through reverse ray tracing, the corresponding object plane dot diagram can be inverted and determined; based on the object plane dot diagram, the dynamic distortion simulation of the near-eye display optical system to be simulated is realized, which can effectively eliminate the influence of the optical distortion of the near-eye display optical system to be simulated itself on the dynamic distortion simulation results, thereby further improving the accuracy of the dynamic distortion simulation results.

[0142] In one embodiment, Figure 8 As shown, Figure 8The present invention is a flow chart of the step of determining a first image plane principal ray incident angle dot pattern in an embodiment; based on optical information, determining a plurality of first image plane principal ray incident angle dot patterns corresponding to a plurality of pupil deviation conditions of an object plane dot pattern comprises the following steps:

[0143] Step S801, obtaining a plurality of first pupil rotation angles and an eye movement function model.

[0144] Among them, the first pupil rotation angle refers to the angle of change of the pupil relative to the optical axis during the simulation process; the eye movement function model is used to describe the movement trajectory of the eye in three-dimensional space; the movement trajectory may include but is not limited to the pupil lateral deviation, the pupil longitudinal deviation, and the pupil rotation angle.

[0145] Step S802: determining a first pupil deviation corresponding to each first pupil rotation angle based on the eye movement function model and a plurality of first pupil rotation angles.

[0146] Among them, the first pupil deviation includes a lateral pupil deviation and a longitudinal pupil deviation; wherein the lateral pupil deviation is the change in pupil displacement in a direction perpendicular to the optical axis; and the longitudinal pupil deviation is the change in pupil displacement along the optical axis.

[0147] Step S803: determining the pupil position corresponding to each first pupil deviation amount according to the multiple first pupil deviation amounts.

[0148] Among them, each pupil posture has a corresponding pupil deviation condition; it should be noted that the pupil is the aperture in the optical system, and the change of the pupil posture will affect the light entering the eye, and the pupil deviation condition refers to the state when the pupil deviates from the optical axis of the optical system. Therefore, each pupil posture will have a corresponding pupil deviation condition.

[0149] Among them, the optical information includes pupil posture. It should be noted that during the simulation process, it is necessary to simulate the human eye to quantify the degree of dynamic distortion of the near-eye display optical system to be simulated. Therefore, it is necessary to simulate the human eye pupil as the diaphragm of the near-eye display optical system to be simulated, and simulate the changes of different pupil postures by adjusting the diaphragm posture, that is, simulate different field of view angles. It can be understood that the pupil posture is the diaphragm posture corresponding to the near-eye display optical system to be simulated.

[0150] Exemplarily, multiple first pupil rotation angles and an eye movement function model are obtained, and based on the eye movement function model, the first pupil deviation corresponding to each first pupil rotation angle, that is, the lateral pupil deviation and the longitudinal pupil deviation, are calculated respectively according to the multiple first pupil rotation angles; and then based on each first pupil deviation, the pupil posture corresponding to each first pupil deviation, that is, the aperture posture, can be determined; according to each pupil posture, the corresponding pupil deviation condition can be determined.

[0151] Step S804, based on the optical information and the object plane dot map, determine the first image plane chief ray incident angle dot map corresponding to each pupil deviation condition.

[0152] Exemplarily, for each pupil deviation condition, that is, each pupil posture, based on the optical information and the object plane dot map corresponding to the object plane, ray tracing can be used to determine the first image plane principal light incidence angle dot map corresponding to each pupil deviation condition and the object plane dot map on the image plane.

[0153] In this embodiment, based on the eye movement function model, the first pupil deviation corresponding to each first pupil rotation angle can be determined, and then according to each first pupil deviation, the corresponding pupil posture can be determined; based on different pupil postures, that is, different pupil deviation conditions, the first image plane principal light incident angle dot matrix corresponding to each pupil deviation condition can be accurately determined, which lays a foundation for improving the accuracy of dynamic distortion simulation results.

[0154] In one embodiment, before obtaining a plurality of first pupil rotation angles and an eye movement function model, the following steps are included:

[0155] Step 1: Obtain a pupil rotation model and multiple second pupil rotation angles.

[0156] The pupil rotation model is used to determine the motion trajectory corresponding to the pupil rotation; the motion trajectory includes the pupil rotation angle, the pupil lateral deviation and the pupil longitudinal deviation. The second pupil rotation angle is the angle of change of the pupil relative to the optical axis position during the establishment of the eye movement function model.

[0157] Step 2: Determine a plurality of corresponding second pupil deviations based on the pupil rotation model and a plurality of second pupil rotation angles.

[0158] The second pupil deviation includes a pupil lateral (X direction) deviation and a pupil longitudinal (Z direction) deviation.

[0159] Step 3: establishing an eye movement function model according to the multiple second pupil rotation angles and the multiple second pupil deviation amounts.

[0160] It is understandable that every time the human eye rotates, that is, when the pupil position deviates from the optical axis, corresponding lateral and longitudinal deviations will be generated. Therefore, by studying the relationship between the second pupil rotation angle and the second pupil deviation, a corresponding eye movement function model can be established.

[0161] For example, see Fig. 9 , Fig. 9The anatomical structure of the human eye is given. The anatomical structure of the human eye can be approximated by two spheres connected together. The outer contour of the eyeball can be represented by a large sphere with a diameter of 24 mm; the iris of the human eye can be represented by a small sphere with a diameter of 11 mm; the distance from the iris to the center of the eyeball is 10.67 mm. Based on the anatomical structure of the human eye, the corresponding pupil rotation model can be obtained. The pupil rotation model is as follows: Fig.10 shown.

[0162] Based on the pupil rotation model, when the second pupil rotation angle is 27.84°, the pupil lateral (X direction) deviation is 5 mm, and the pupil longitudinal (Z direction) deviation is 1.24 mm. Similarly, the pupil lateral deviation and pupil longitudinal deviation corresponding to different second pupil rotation angles can be obtained. The specific relationship between the second pupil rotation angle and the second pupil deviation is shown in Table 1. Furthermore, based on multiple second pupil rotation angles and multiple second pupil deviations, a corresponding eye movement function model can be established. The function expression of the eye movement function model is shown in formulas (2)-(3); the schematic diagram of the eye movement function model is shown in Fig.11 shown.

[0163] x=|1.333×10 -5 ×β 3 +2×10 -4 ×β 2 +0.1833×β+0.01| (2)

[0164] z=|-2.0919×10 -19 ×β 3 +0.0016×β 2 -1.2322×10 -16 ×β+5.4616 -16 |(3)

[0165] Among them, x represents the lateral deviation of the pupil, in mm; β represents the second pupil rotation angle, in deg; z represents the longitudinal deviation of the pupil, in mm.

[0166] Table 1 Relationship between the second pupil rotation angle and the second pupil deviation

[0167]

[0168]

[0169] In this embodiment, based on the pupil rotation model and multiple second pupil rotation angles, the corresponding multiple second pupil deviations can be accurately determined, and then an eye movement function model is established according to the second pupil rotation angles and the second pupil deviations, laying the foundation for achieving different pupil deviation conditions.

[0170] In one embodiment, based on the optical information and the object plane dot map, determining the first image plane chief ray incident angle dot map corresponding to each pupil deviation condition includes the following steps:

[0171] Step 1: for each of the multiple pupil deviation conditions, based on the first optical information and the object plane dot pattern, determine the first principal ray incident angle dot pattern corresponding to each pupil deviation condition.

[0172] Among them, the optical information includes first optical information and second optical information; the first optical information refers to the optical information corresponding to the near-eye display optical system to be simulated in the absence of target surface defect information; the second optical information refers to the optical information corresponding to the near-eye display optical system to be simulated in the presence of target surface defect information; wherein the target surface defect information is used to simulate defects existing on the optical surface in the near-eye display optical system to be simulated, such as scratches, pitting and other defects on the optical surface.

[0173] Among them, the first chief ray incident angle dot pattern refers to the chief ray incident angle dot pattern of the object plane dot pattern on the image plane corresponding to the pupil deviation condition when there is no target surface defect information in the near-eye display optical system to be simulated.

[0174] Step 2: Determine the second principal ray incident angle dot pattern corresponding to each pupil deviation condition based on the second optical information and the object plane dot pattern.

[0175] Among them, the second chief ray incident angle dot pattern refers to the chief ray incident angle dot pattern of the object plane dot pattern on the image plane corresponding to the pupil deviation condition when there is target surface defect information in the near-eye display optical system to be simulated.

[0176] Step 3, obtaining the first image plane chief ray incident angle dot pattern corresponding to each deviated pupil condition according to the first chief ray incident angle dot pattern and the second chief ray incident angle dot pattern corresponding to each deviated pupil condition.

[0177] Exemplarily, based on the first optical information, i.e., the optical information without the target surface defect information, the first principal ray incident angle dot diagram of the object plane dot diagram on the image plane is determined by ray tracing under each eccentric pupil condition; based on the second optical information, i.e., the optical information with the target surface defect information, the second principal ray incident angle dot diagram of the object plane dot diagram on the image plane under each eccentric pupil condition is determined by ray tracing, and then the first principal ray incident angle dot diagram and the second principal ray incident angle dot diagram corresponding to each eccentric pupil condition are used to construct the first image plane principal ray incident angle dot diagram corresponding to each eccentric pupil condition.

[0178] In this embodiment, the first optical information and the second optical information can be used to respectively determine the first principal ray incident angle dot diagram of the object plane dot diagram on the image plane when there is no target surface defect information, and the first principal ray incident angle dot diagram of the object plane dot diagram on the image plane when there is target surface defect information. Based on this, the foundation is laid for realizing the simulation of dynamic distortion.

[0179] In one embodiment, before determining the second chief ray incident angle dot pattern corresponding to each pupil deviation condition based on the second optical information and the object plane dot pattern, the following steps are also included:

[0180] Step 1: Obtain preset position and surface defect data.

[0181] The preset position is located within a preset surface of the near-eye display optical system to be simulated, and the preset surface is any optical surface within the near-eye display optical system to be simulated, which is not specifically limited here.

[0182] The surface defect data may be, but is not limited to, generated by a Gaussian function, which is not specifically limited here; for example, a Gaussian defect, namely a convex hull, is generated based on a Gaussian function, and the data corresponding to the Gaussian defect is the surface defect data.

[0183] Step 2: Convert the format of the surface defect data to determine the target surface defect information.

[0184] Step 3: Import the target surface defect information into a preset position to determine the second optical information corresponding to the near-eye display optical system to be simulated.

[0185] In an exemplary embodiment, when implementing dynamic distortion simulation based on Codev software, it is necessary to first import the first optical information of the near-eye display optical system to be simulated into the Codev software; if the second optical information is to be obtained, the following steps need to be performed: obtain the preset position and surface defect data, convert the format of the surface defect data into an int file, obtain the target surface defect information, and then use the import function of the interference pattern file in the Codev software to import the target surface defect information into the preset position of the preset surface in the near-eye display optical system to be simulated. Based on this, the second optical information corresponding to the near-eye display optical system to be simulated can be obtained.

[0186] In this embodiment, by converting the format of the surface defect data, the corresponding target surface defect information can be determined; then, the target surface defect information can be imported into a preset position to determine the second optical information corresponding to the near-eye display optical system to be simulated, thereby realizing the simulation of optical surface defects.

[0187] In one embodiment, Fig.12 As shown, Fig.12The schematic diagram of the flow of determining the second image plane principal ray incident angle dot pattern step in one embodiment; based on the optical information, determining the second image plane principal ray incident angle dot pattern corresponding to the object plane dot pattern under the condition of no biased pupil includes:

[0188] Step S1201, based on the first optical information and the object plane dot pattern, determine the third principal ray incident angle dot pattern corresponding to the unbiased pupil condition.

[0189] Among them, the third chief ray incident angle dot diagram refers to the chief ray incident angle dot diagram of the object plane dot diagram on the image plane corresponding to the unbiased pupil condition when there is no target surface defect information in the near-eye display optical system to be simulated.

[0190] Step S1202, based on the second optical information and the object plane dot pattern, determine the fourth principal ray incident angle dot pattern corresponding to the unbiased pupil condition.

[0191] Among them, the fourth chief ray incident angle dot pattern refers to the chief ray incident angle dot pattern of the object plane dot pattern on the image plane corresponding to the unbiased pupil condition when there is target surface defect information in the near-eye display optical system to be simulated.

[0192] Step S1203, obtaining a second image plane chief ray incident angle dot pattern according to the third chief ray incident angle dot pattern and the fourth chief ray incident angle dot pattern.

[0193] Exemplarily, based on the first optical information, i.e., the optical information without the target surface defect information, the third principal ray incident angle dot diagram of the object plane dot diagram on the image plane under the unbiased pupil condition is determined by ray tracing; based on the second optical information, i.e., the optical information with the target surface defect information, the fourth principal ray incident angle dot diagram of the object plane dot diagram on the image plane under the unbiased pupil condition is determined by ray tracing, and then the second image plane principal ray incident angle dot diagram corresponding to the unbiased pupil condition is constructed by the third principal ray incident angle dot diagram and the fourth principal ray incident angle dot diagram.

[0194] In this embodiment, the first optical information and the second optical information can be used to respectively determine the third principal ray incident angle dot diagram of the object plane dot diagram on the image plane when there is no target surface defect information, and the fourth principal ray incident angle dot diagram of the object plane dot diagram on the image plane when there is target surface defect information. Based on this, the foundation is laid for realizing the simulation of dynamic distortion.

[0195] In one embodiment, Fig.13 As shown, Fig.13The present invention is a flow chart of the steps of determining the dynamic distortion simulation result of the near-eye display optical system to be simulated in one embodiment; based on a plurality of first image plane principal ray incident angle dot matrix diagrams and a second image plane principal ray incident angle dot matrix diagram, determining the dynamic distortion simulation result of the near-eye display optical system to be simulated includes the following steps:

[0196] Step S1301, for each of the multiple pupil deviation conditions, determine the first simulation result corresponding to each pupil deviation condition according to the first principal ray incident angle dot matrix diagram corresponding to each pupil deviation condition and the third principal ray incident angle dot matrix diagram.

[0197] It should be noted that the first chief ray incident angle dot diagram and the third chief ray incident angle dot diagram are both image plane chief ray incident angle dot diagrams determined in the absence of target surface defect information.

[0198] The first simulation result is the change in the incident angle of the principal ray corresponding to each pupil deviation condition when there is no target surface defect information. It should be noted that the first simulation result includes a plurality of data on the change in the incident angle of the principal ray.

[0199] Exemplarily, according to the first principal ray incident angle dot diagram corresponding to each deviated pupil condition and the third principal ray incident angle dot diagram, the first principal ray incident angle dot diagram corresponding to each deviated pupil condition is subtracted from the third principal ray incident angle dot diagram to determine the first simulation result corresponding to each deviated pupil condition. Optionally, the first simulation result can be represented in the form of a heat map, but is not limited to it, which is not specifically limited here.

[0200] Step S1302, determining a second simulation result corresponding to each deviated pupil condition according to the second principal ray incident angle dot pattern corresponding to each deviated pupil condition and the fourth principal ray incident angle dot pattern.

[0201] It should be noted that the second chief ray incident angle dot diagram and the fourth chief ray incident angle dot diagram are both image plane chief ray incident angle dot diagrams determined in the presence of target surface defect information.

[0202] The second simulation result is the change in the incident angle of the main ray corresponding to each pupil deviation condition when there is target surface defect information. It should be noted that the second simulation result includes a plurality of data on the change in the incident angle of the main ray.

[0203] Exemplarily, according to the second principal ray incident angle dot diagram corresponding to each deviated pupil condition and the fourth principal ray incident angle dot diagram, the second principal ray incident angle dot diagram corresponding to each deviated pupil condition is subtracted from the fourth principal ray incident angle dot diagram to determine the second simulation result corresponding to each deviated pupil condition. Optionally, the second simulation result can be represented in the form of a heat map, but is not limited to it, which is not specifically limited here.

[0204] Step S1303, determining a dynamic distortion simulation result of the near-eye display optical system to be simulated according to the first simulation result and the second simulation result corresponding to each pupil deviation condition.

[0205] The dynamic distortion simulation result includes the position, shape and degree of the dynamic distortion; preferably, the degree of distortion is quantitatively characterized by the peak value of the change in the incident angle of the main light. Optionally, the dynamic distortion simulation result can be represented in the form of a heat map, but is not limited to it, wherein the horizontal and vertical coordinates of the heat map are the horizontal field of view and the vertical field of view of the near-eye display optical system to be simulated, respectively.

[0206] It should be noted that the dynamic distortion of the near-eye display optical system to be simulated usually includes two types. One is the global dynamic distortion introduced by the aberration of the near-eye display optical system to be simulated; the other is the local dynamic distortion introduced by the optical surface defects of the near-eye display optical system to be simulated. Among them, the local dynamic distortion is the main cause of the jitter and jump of the virtual video image, and the human eye is extremely sensitive to it. Therefore, this embodiment focuses on simulating local dynamic distortion.

[0207] It can be understood that the first simulation result is the global dynamic distortion simulation result; the second simulation result is the comprehensive simulation result of the global dynamic distortion and the local dynamic distortion. Therefore, according to the first simulation result and the second simulation result corresponding to each pupil deviation condition, the second simulation result can be subtracted from the first simulation result, and the peak value of the difference can be used as the dynamic distortion simulation result of the near-eye display optical system to be simulated, wherein the dynamic distortion simulation result is the local dynamic distortion simulation result.

[0208] In this embodiment, based on the first simulation result and the second simulation result corresponding to each pupil deviation condition, the dynamic distortion simulation result of the near-eye display optical system to be simulated can be accurately quantified, thereby improving the accuracy of the dynamic distortion simulation result.

[0209] In one embodiment, Fig.14 As shown, Fig.14 FIG. 4 is a flow chart of a dynamic distortion simulation method in another embodiment; the dynamic distortion simulation method further includes the following steps:

[0210] Step S1401, for each of the multiple pupil deviation conditions, determine the corresponding second image plane image height dot map according to the first principal ray incident angle dot map corresponding to each pupil deviation condition.

[0211] The second image plane image height dot matrix includes multiple image height values; the image height value refers to the distance from the image point to the center of the image plane. The second image plane image height dot matrix is ​​the image plane image height dot matrix of the object plane dot matrix on the image plane when there is no target surface defect information.

[0212] Step S1402, determining the corresponding third image plane image height dot map according to the second chief ray incident angle dot map corresponding to each pupil deviation condition.

[0213] Among them, the third image plane image height dot matrix includes multiple image height values; the third image plane image height dot matrix is ​​the image plane image height dot matrix of the object plane dot matrix on the image plane when there is target surface defect information.

[0214] It should be noted that, based on formula (1), the first principal ray incident angle dot diagram and the second principal ray incident angle dot diagram are processed respectively to determine the corresponding second image plane image height dot diagram and third image plane image height dot diagram.

[0215] Step S1403, generating a text video according to the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition.

[0216] Among them, the text video is used to simulate the imaging condition of the near-eye display optical system to be simulated observed when the human eye moves.

[0217] It should be noted that, based on the second image plane image height dot map and the third image plane image height dot map corresponding to each pupil deviation condition, a corresponding mapping text image can be generated, and then multiple mapping text images can be synthesized to generate a text video.

[0218] Step S1404: establishing a dynamic distortion perception model based on the text video.

[0219] It should be noted that, in an exemplary embodiment, 10 users with corrected visual acuity of 1.0 are recruited to observe the text video, and the text video is evaluated according to the grading rules. Among them, the grading rules are set according to the human eye completely invisible, there will be an impact but acceptable, and unacceptable, and the grades can be expressed in digital form such as 1, 2, and 3. Further, the evaluation results of all users on the text video are counted, and the mean of the evaluation results of all users on the text video is calculated to obtain the dynamic distortion perception result of the text video. Similarly, according to the method recorded in the above embodiment, text videos under different target surface defect information are generated respectively, and the dynamic distortion perception results and dynamic distortion simulation results under each target surface defect information are determined. Further, according to the relationship between the dynamic distortion perception results and the dynamic distortion simulation results under each target surface defect information, a dynamic distortion perception model is established.

[0220] Preferably, the dynamic distortion perception model is sent to the first terminal, so that the first terminal outputs the user's perception result of the dynamic distortion of the near-eye display optical system according to the dynamic distortion perception model.

[0221] The first terminal may be, but is not limited to, a dynamic distortion test system, which is not specifically limited herein.

[0222] It can be understood that when the first terminal is a dynamic distortion test system, when the dynamic distortion test of the near-eye display optical system is implemented based on the dynamic distortion test system, the dynamic distortion perception model can directly output the user's perception results of the dynamic distortion of the near-eye display optical system, thereby realizing the quantitative perceptual evaluation of dynamic distortion.

[0223] In this embodiment, based on the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition, a text video corresponding to the target surface defect information can be generated, and then based on the text videos corresponding to different target surface defect information, a dynamic distortion perception model can be established, thereby realizing the quantitative perceptual evaluation of dynamic distortion.

[0224] In one embodiment, generating a text video according to the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition includes the following steps:

[0225] Step 1: Get the original text image.

[0226] Step 2, mapping the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each pupil deviation condition, and determining the overall homography matrix corresponding to each pupil deviation condition.

[0227] Step 3: According to the overall homography matrix corresponding to each eccentric pupil condition, the original text image is mapped to obtain a mapped text image corresponding to each eccentric pupil condition.

[0228] Step 4: Generate a text video based on the mapped text images corresponding to multiple pupil deviation conditions.

[0229] The original text image refers to an image containing readable text content; for example, the original text image is Fig.15 As shown, it can be understood that Fig.15 is the original text image without distortion. The overall homography matrix is ​​a geometric transformation matrix used to map the original text image to obtain the corresponding mapped text image, and at the same time ensure that the collinearity and intersection-over-union ratio of the midpoints of the original text image and the mapped text image remain unchanged.

[0230] For example, Fig.15 Take the original text image in as an example to illustrate, assuming that the target surface defect information is as follows Fig.16 As shown, Fig.16The target surface defect information in the simulation is imported into the center position of the preset surface of the near-eye display optical system to be simulated, and the second optical information corresponding to the near-eye display optical system to be simulated is determined; based on the second optical information, the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each deviation pupil condition are determined; further, the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each deviation pupil condition are mapped to determine the overall homography matrix corresponding to each deviation pupil condition; and then based on the overall homography matrix corresponding to each deviation pupil condition, Fig.15 The original text image in is mapped to obtain a mapped text image corresponding to each pupil deviation condition; for example, one of the multiple pupil deviation conditions is taken as an example, and the mapped text image corresponding to the pupil deviation condition is as follows: Fig.17 As shown. Further, the mapped text images corresponding to the multiple pupil deviation conditions are synthesized to generate a text video. It can be understood that Fig.17 It can be seen that when the center position of the preset surface of the near-eye display optical system to be simulated exists Fig.16 When the target surface defect information is obtained, the center position of the mapped text image will produce corresponding distortion.

[0231] In this embodiment, by mapping the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each deviated pupil condition, the overall homography matrix corresponding to each deviated pupil condition can be accurately determined; then based on multiple overall homography matrices, mapped text images corresponding to multiple deviated pupil conditions can be obtained to generate corresponding text videos. Based on this, the imaging condition of the near-eye display optical system to be simulated observed by the human eye when the eyeball rotates can be accurately simulated, laying the foundation for establishing a dynamic distortion perception model.

[0232] In one embodiment, mapping the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each pupil deviation condition to determine the overall homography matrix corresponding to each pupil deviation condition includes the following steps:

[0233] Step 1: divide the second image plane image height dot matrix corresponding to each pupil deviation condition into regions to obtain a first number of first divided sub-regions.

[0234] The first number needs to be determined based on actual interval division rules and is not specifically limited here.

[0235] Optionally, the method of dividing the second image plane image height dot map corresponding to each deviated pupil condition into regions may be, but is not limited to: based on the triangulation method, a pair of triangles is generated for every four adjacent points on the second image plane image height dot map corresponding to each deviated pupil condition, to obtain corresponding multiple first divided sub-regions. It is understandable that each first divided interval includes a pair of triangles. In an exemplary embodiment, a schematic diagram of dividing the second image plane image height dot map corresponding to one of the multiple deviated pupil conditions into regions is shown in FIG. Fig.18 shown.

[0236] Step 2: Perform local mapping on a first number of first divided sub-regions to determine a corresponding plurality of first local homography mapping matrices.

[0237] Optionally, the method of performing local mapping on the first number of first divided sub-regions may be but is not limited to: performing local mapping on each of the first number of first divided sub-regions through affine transformation to obtain a first local homography mapping matrix corresponding to each first divided sub-region.

[0238] Step 3, dividing the third image plane image height dot matrix corresponding to each pupil deviation condition into regions to obtain a second number of second divided sub-regions.

[0239] Among them, the second number is equal to the first number. It should be noted that the method of dividing the third image plane image height dot matrix corresponding to each deviated pupil condition into regions needs to be the same as the method of dividing the second image plane image height dot matrix corresponding to each deviated pupil condition into regions, so as to ensure accurate and reliable generation of the overall homography matrix corresponding to each deviated pupil condition.

[0240] Optionally, the method for dividing the region of the third image plane image height dot map corresponding to each deviated pupil condition may be, but is not limited to: based on the triangulation method, a pair of triangles is generated for every four adjacent points on the third image plane image height dot map corresponding to each deviated pupil condition to obtain a corresponding plurality of second divided sub-regions. It is understandable that each second divided interval includes a pair of triangles. In an exemplary embodiment, based on Fig.16 The target surface defect information shown is a schematic diagram of dividing the area of ​​the third image plane image height dot matrix corresponding to one of the multiple pupil deviation conditions, as shown in Fig.19 shown.

[0241] Step 4: Perform local mapping on a second number of second divided sub-regions to determine a corresponding plurality of second local homography mapping matrices.

[0242] Optionally, the method of performing local mapping on the second number of second divided sub-regions may be but is not limited to: performing local mapping on each of the second number of second divided sub-regions through affine transformation to obtain a second local homography mapping matrix corresponding to each second divided sub-region.

[0243] Step 5: Determine the overall homography matrix corresponding to each skewed pupil condition according to the multiple first local homography mapping matrices and the multiple second local homography mapping matrices corresponding to each skewed pupil condition.

[0244] Preferably, step 5, determining the overall homography matrix corresponding to each skewed pupil condition according to the multiple first local homography mapping matrices and the multiple second local homography mapping matrices corresponding to each skewed pupil condition, comprises the following steps:

[0245] Step 5.1, determining the relative position of each first divided sub-region in the image height dot matrix of the second image plane.

[0246] Step 5.2, determining the relative position of each second divided sub-region in the image height dot matrix of the third image plane.

[0247] Step 5.3, performing spatial distortion removal processing on the first local homography mapping matrix corresponding to the first divided sub-region and the second local homography mapping matrix corresponding to the second divided sub-region with the same relative position, to obtain an intermediate local homography mapping matrix corresponding to each relative position.

[0248] The method of removing the spatial distortion needs to be set according to actual needs and is not specifically limited here.

[0249] Step 5.4, determining the corresponding overall homography matrix based on multiple intermediate local homography mapping matrices.

[0250] It should be noted that in addition to dynamic distortion, there will also be spatial distortion when the eyeball moves. Therefore, in order to ensure that the text video can accurately simulate the imaging of the near-eye display optical system to be simulated observed when the human eye moves, it is necessary to eliminate the impact of spatial distortion on the accuracy of the text video.

[0251] For example, Fig.18 and Fig.19 Take this as an example to illustrate. Fig.18 and Fig.19Corresponding to the skewed pupil condition, determine the relative position of each first divided sub-region in the second image plane image height dot matrix under the current skewed pupil condition, and the relative position of each second divided sub-region in the third image plane image height dot matrix; further, perform matrix division operation on the first local homography mapping matrix corresponding to the first divided sub-regions with the same relative position under the current skewed pupil condition and the second local homography mapping matrix corresponding to the second divided sub-region to eliminate the spatial distortion corresponding to the current skewed pupil condition, and obtain the intermediate local homography mapping matrix corresponding to each relative position under the current skewed pupil condition; and then determine the overall homography matrix corresponding to the current skewed pupil condition based on multiple intermediate local homography mapping matrices.

[0252] In this embodiment, by performing spatial distortion removal processing on the first local homography mapping matrix and the second local homography mapping matrix with the same relative position, an intermediate local homography mapping matrix corresponding to each relative position is obtained, which can effectively eliminate the influence of spatial distortion on the mapped text image, thereby improving the accuracy of the text video.

[0253] In one embodiment, the dynamic distortion simulation method further includes:

[0254] Based on the dynamic distortion simulation results of the near-eye display optical system to be simulated and the target surface defect information, an objective function model is established; the objective function model is used to determine the correlation between dynamic distortion and surface defects;

[0255] The objective function model is sent to the second terminal, so that the second terminal outputs the sensitivity of the near-eye display optical system to surface defects according to the objective function model.

[0256] It should be noted that there are corresponding dynamic distortion simulation results for target surface defect information of different degrees, and based on the relationship between the target surface defect information and the dynamic distortion simulation results, the corresponding objective function model can be determined.

[0257] The second terminal may be, but is not limited to, a design terminal corresponding to the near-eye display optical system, which is not specifically limited here. It is understandable that in the design stage of the near-eye display optical system, based on the objective function model, the sensitivity analysis of the near-eye display optical system to surface defects can be realized, which lays the foundation for improving the performance of the near-eye display optical system.

[0258] Exemplarily, based on target surface defect information of different degrees and dynamic distortion simulation results of the near-eye display optical system to be simulated corresponding to the target surface defect information of different degrees, an objective function model is established; then, in the design stage of the near-eye display optical system, the objective function model is sent to the design end corresponding to the near-eye display optical system, so that the design end performs sensitivity analysis on the near-eye display optical system according to the objective function model, thereby outputting the sensitivity results of the near-eye display optical system to surface defects. It should be noted that the defect evaluation corresponding to the target surface defect information is: defect peak value under unit size, that is, dpeak / dsize; wherein dpeak is the defect peak value; dsize is the unit size; the defect peak values ​​under unit size corresponding to target surface defect information of different degrees are different from each other.

[0259] In this embodiment, there are corresponding dynamic distortion simulation results based on target surface defect information of different degrees, and then based on the relationship between the target surface defect information and the dynamic distortion simulation results, the corresponding objective function model can be determined, and then based on the objective function model, the sensitivity analysis of the near-eye display optical system to surface defects can be realized, laying the foundation for improving the performance of the near-eye display optical system.

[0260] In a specific embodiment, see Fig. 20 , taking the dynamic distortion simulation method applied to Codev software as an example for explanation, including the following steps:

[0261] Step S2001, obtaining an object plane dot pattern and first optical information and second optical information corresponding to a near-eye display optical system to be simulated.

[0262] The first optical information refers to the optical information corresponding to the near-eye display optical system to be simulated when there is no target surface defect information, and the second optical information refers to the optical information corresponding to the near-eye display optical system to be simulated when there is target surface defect information.

[0263] Step S2002, based on the first optical information and the second optical information, determining a plurality of first principal ray incident angle dot diagrams and a plurality of second principal ray incident angle dot diagrams corresponding to a plurality of pupil deviation conditions of the object plane dot diagram.

[0264] Step S2003, based on the first optical information and the second optical information, determining the third principal ray incident angle dot diagram and the fourth principal ray incident angle dot diagram corresponding to the object plane dot diagram under the unbiased pupil condition.

[0265] Step S2004, for each of the multiple pupil deviation conditions, determine the first simulation result corresponding to each pupil deviation condition based on the first principal ray incident angle dot matrix diagram corresponding to each pupil deviation condition and the third principal ray incident angle dot matrix diagram.

[0266] Step S2005, determining the second simulation result corresponding to each deviated pupil condition according to the second principal ray incident angle dot pattern corresponding to each deviated pupil condition and the fourth principal ray incident angle dot pattern.

[0267] Step S2006, determining a third simulation result corresponding to each deviated pupil condition according to the difference between the second simulation result corresponding to each deviated pupil condition and the first simulation result.

[0268] Step S2007, determining a dynamic distortion simulation result of the near-eye display optical system to be simulated according to the third simulation results corresponding to the plurality of pupil deviation conditions.

[0269] In a specific embodiment, the dynamic distortion simulation method is applied to a first terminal as an example for description, wherein the first terminal is a dynamic distortion test system.

[0270] It should be noted that, see Fig.21 The dynamic distortion test device for near-eye display optical system includes: an operating platform and an equipment platform; the operating platform includes a computer display screen, a material area, and a material tray for testing OK and NG. The equipment platform adopts a double-station mode, and both stations contain lighting fixtures, a six-degree-of-freedom (Rx / Ry / Rz / X / Y / Z) adjustment platform, a transport electric axis, an internal focusing autocollimator, and a dynamic distortion test system; the dynamic distortion test system includes an anthropomorphic eye lens and a large target test camera.

[0271] Exemplarily, the dynamic distortion test process includes the following steps:

[0272] Step 1: The operator moves the module to be tested to the lighting fixture and fixes it.

[0273] Among them, the module to be tested is the near-eye display optical system to be tested.

[0274] Step 2: Move the module to be tested to the bottom of the internal focusing autocollimator by moving the electric axis, detect the posture of the module to be tested, and use the six-degree-of-freedom adjustment platform to adjust the posture of the module to be tested to ensure the consistency of the posture of the loading module.

[0275] Step 3: Move the module to be tested to the dynamic distortion test system by moving the electric shaft, perform a dynamic distortion test on the module to be tested, and obtain the test results.

[0276] Step 4: The transport electric axis moves the module to be tested to the loading and unloading position, the operator unloads the module, and places the module into the OK or NG tray according to the test results.

[0277] It should be noted that in order to ensure the accuracy of the dynamic distortion test results, it is necessary to test the optical axis of the module to be tested based on the reflective eccentricity test method using an internal focusing autocollimator; and adjust the posture of the module to be tested based on the six-degree-of-freedom adjustment platform so that the optical axis of the module to be tested coincides with the optical axis of the internal focusing autocollimator; further, move the module to be tested to the bottom of the dynamic distortion test system, adjust the tilt (i.e. inclination) and X / Y of the module to be tested to compensate for the calibration deviation between the internal focusing autocollimator test station and the dynamic distortion test station, so that the optical axis of the module to be tested coincides with the optical axis of the dynamic distortion test station; further, use the six-degree-of-freedom adjustment platform to simulate the movement of the human eye, collect images with different pupil deviation conditions (i.e., the image plane main light incidence angle dot matrix), and then based on the image processing method, combined with the principle of the dynamic distortion simulation method, calculate the dynamic distortion result corresponding to the module to be tested. The specific calculation principle is the same as the method principle recorded in the above embodiment, and will not be repeated here.

[0278] It should be noted that before testing the dynamic distortion of the module to be tested based on the near-eye display optical system dynamic distortion test device, the dynamic distortion test station and the internal focusing autocollimator test station need to be calibrated separately to ensure the accuracy of the test results.

[0279] For example, the schematic diagram of the dynamic distortion test station calibration scheme is as follows: Fig. 22 As shown, Fig. 22 In the figure, Cono means humanoid eye lens; Lens set means module; Display means cross lighting; the calibration scheme of dynamic distortion test station includes the following steps:

[0280] Step 1: Place the module on the lighting fixture and move it to the dynamic distortion test station. Adjust the Z-axis height of the six-degree-of-freedom adjustment platform to ensure that the distance from the Cono aperture of the anthropomorphic eye lens to the vertex of the module surface is 15 mm.

[0281] Step 2: Move the module to the dynamic distortion test station and light up the cross (the cross light chart is as shown in the figure below). Fig.23 As shown), adjust the XY axis of the six-degree-of-freedom adjustment platform to make the cross image relatively clear in the anthropomorphic eye lens Cono.

[0282] Step 3: Based on the current cross image, adjust the exposure of the humanoid eye lens Cono so that the maximum grayscale value of the image is 170±15.

[0283] Step 4, adjust the plane inclination UV of the six-degree-of-freedom adjustment platform and the inclination W around the Z axis so that the center of the cross image is located at the center of the anthropomorphic eye lens Cono of the dynamic distortion test system.

[0284] Step 5: Light up the module ghost calibration pattern (ghost calibration pattern card is as follows Fig.24As shown), adjust the exposure and gain of the anthropomorphic eye lens Cono so that the anthropomorphic eye lens Cono can clearly see the ghost image.

[0285] Step 6: Fine-tune the X-axis of the six-degree-of-freedom adjustment platform and the Y-axis of the transport motor shaft so that the ghost image and the true image are aligned. Repeat. Fig. 22 Repeat ③ and ④ at least once until no adjustment is needed. Based on the above steps 1-6, the calibration of the dynamic distortion test station can be achieved.

[0286] For example, the schematic diagram of the calibration scheme of the internal focusing autocollimator test station is as follows: Fig.25 As shown, since the reference axis of the dynamic distortion test station is the dynamic distortion test system, the position of the jig has been calibrated when the dynamic distortion test station is calibrated. The calibration of the internal focusing autocollimator test station uses the position of the jig as a reference to calibrate the internal focusing autocollimator and the reflector 2. The specific calibration steps include: focusing the internal focusing autocollimator to infinity, placing a flat crystal on the jig, and adjusting the plane inclination U / V of the reflector 2 so that the cross image reflected by the flat crystal is located at the center of the chip of the internal focusing autocollimator.

[0287] The above-mentioned dynamic distortion simulation method, in the first aspect, can accurately determine the multiple first image plane principal ray incident angle dot patterns corresponding to multiple deviated pupil conditions and the second image plane principal ray incident angle dot patterns corresponding to the non-deviated pupil condition based on the optical information; based on the first image plane principal ray incident angle dot pattern and the second image plane principal ray incident angle dot pattern, the dynamic distortion simulation result of the near-eye display optical system to be simulated is determined, which can fully consider the adaptability of the near-eye display optical system to be simulated and the human eye, and realize the quantification of the dynamic distortion of the near-eye display optical system to be simulated from the perspective of the human eye, thereby improving the accuracy of the dynamic distortion simulation result of the near-eye display optical system. In the second aspect, based on the second image plane image height dot pattern and the third image plane image height dot pattern corresponding to each deviated pupil condition, a text video corresponding to the target surface defect information can be generated, and then based on the text videos corresponding to different target surface defect information, a dynamic distortion perception model can be established, thereby realizing the quantitative perception evaluation of dynamic distortion. Thirdly, based on the relationship between the target surface defect information and the dynamic distortion simulation results, the corresponding objective function model can be determined. Then, based on the objective function model, the sensitivity analysis of the near-eye display optical system to surface defects can be realized, laying the foundation for improving the performance of the near-eye display optical system.

[0288] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0289] Based on the same inventive concept, the embodiment of the present application also provides a dynamic distortion simulation device for implementing the dynamic distortion simulation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more dynamic distortion simulation device embodiments provided below can refer to the limitations on the dynamic distortion simulation method above, and will not be repeated here.

[0290] In an exemplary embodiment, Fig.26 As shown, a dynamic distortion simulation device is provided, comprising: an acquisition module 2601, a first determination module 2602, a second determination module 2603 and a dynamic distortion simulation module 2604;

[0291] An acquisition module 2601 is used to acquire an object plane dot pattern and optical information corresponding to a near-eye display optical system to be simulated;

[0292] A first determination module 2602 is used to determine, based on optical information, a plurality of first image plane chief ray incident angle dot diagrams corresponding to the object plane dot diagram under a plurality of deviated pupil conditions;

[0293] A second determination module 2603 is used to determine, based on the optical information, a second image plane principal ray incident angle dot pattern corresponding to the object plane dot pattern under the condition of unbiased pupil;

[0294] The dynamic distortion simulation module 2604 is used to determine the dynamic distortion simulation result of the near-eye display optical system to be simulated based on multiple first image plane principal ray incident angle dot matrix diagrams and second image plane principal ray incident angle dot matrix diagrams.

[0295] The above-mentioned dynamic distortion simulation device, based on optical information, can accurately determine multiple first image plane principal ray incident angle dot patterns corresponding to multiple deviated pupil conditions, and the second image plane principal ray incident angle dot pattern corresponding to the undeviated pupil condition; based on the first image plane principal ray incident angle dot pattern and the second image plane principal ray incident angle dot pattern, the dynamic distortion simulation result of the near-eye display optical system to be simulated is determined, which can fully consider the adaptability of the near-eye display optical system to be simulated and the human eye, realize the quantification of the dynamic distortion of the near-eye display optical system to be simulated from the perspective of the human eye, and improve the accuracy of the dynamic distortion simulation result of the near-eye display optical system.

[0296] In one embodiment, the acquisition module 2601 is also used to

[0297] Obtaining a preset image plane principal ray incident angle dot matrix diagram;

[0298] Performing image height processing on a preset image plane principal ray incident angle dot matrix to determine a corresponding first image plane image height dot matrix;

[0299] Ray tracing is performed on the first image plane image height dot map to determine the object plane dot map.

[0300] In one embodiment, the preset image plane principal ray incident angle dot matrix includes a plurality of principal ray incident angle data; the acquisition module 2601 is also used for

[0301] Performing image height processing on a plurality of principal ray incident angle data to obtain a plurality of corresponding image height values;

[0302] According to the multiple image height values, an image height dot matrix of the first image plane is determined.

[0303] In one embodiment, the optical information includes pupil position; the first determination module 2602 is also used to

[0304] Acquire multiple first pupil rotation angles and eye movement function models;

[0305] Based on the eye movement function model and the plurality of first pupil rotation angles, determining a first pupil deviation amount corresponding to each first pupil rotation angle;

[0306] According to the plurality of first pupil deviation amounts, determining the pupil position corresponding to each first pupil deviation amount; each pupil position has a corresponding pupil deviation condition;

[0307] Based on the optical information and the object plane dot pattern, the first image plane chief ray incident angle dot pattern corresponding to each pupil deviation condition is determined.

[0308] In one embodiment, the dynamic distortion simulation device further includes: a model building module;

[0309] Model building module for

[0310] Obtaining a pupil rotation model and a plurality of second pupil rotation angles;

[0311] Based on the pupil rotation model and the plurality of second pupil rotation angles, determining a plurality of corresponding second pupil deviation amounts;

[0312] An eye movement function model is established according to the multiple second pupil rotation angles and the multiple second pupil deviation amounts.

[0313] In one embodiment, the optical information includes first optical information and second optical information; the first determination module 2602 is further used to

[0314] For each of the multiple pupil deviation conditions, a first principal ray incident angle dot pattern corresponding to each pupil deviation condition is determined based on the first optical information and the object plane dot pattern; the first optical information refers to the optical information corresponding to the near-eye display optical system to be simulated in the absence of target surface defect information;

[0315] Based on the second optical information and the object plane dot map, a second principal ray incident angle dot map corresponding to each pupil deviation condition is determined; the second optical information refers to the optical information corresponding to the near-eye display optical system to be simulated in the presence of target surface defect information;

[0316] According to the first principal ray incident angle dot matrix diagram and the second principal ray incident angle dot matrix diagram corresponding to each deviated pupil condition, the first image plane principal ray incident angle dot matrix diagram corresponding to each deviated pupil condition is obtained.

[0317] In one embodiment, the first determining module 2602 is further configured to:

[0318] Acquiring a preset position and surface defect data; the preset position is located within a preset surface of the near-eye display optical system to be simulated;

[0319] Convert the format of surface defect data to determine target surface defect information;

[0320] The target surface defect information is imported into a preset position to determine the second optical information corresponding to the near-eye display optical system to be simulated.

[0321] In one embodiment, the optical information includes first optical information and second optical information; the second determination module 2603 is further used to

[0322] Determine a third principal ray incident angle dot pattern corresponding to an unbiased pupil condition based on the first optical information and the object plane dot pattern;

[0323] Determine a fourth principal ray incident angle dot pattern corresponding to an unbiased pupil condition based on the second optical information and the object plane dot pattern;

[0324] According to the third principal ray incident angle dot pattern and the fourth principal ray incident angle dot pattern, a second image plane principal ray incident angle dot pattern is obtained.

[0325] In one embodiment, the dynamic distortion simulation module 2604 is also used to

[0326] For each of the plurality of deviated pupil conditions, determine a first simulation result corresponding to each deviated pupil condition according to a first principal ray incident angle dot pattern corresponding to each deviated pupil condition and a third principal ray incident angle dot pattern;

[0327] Determine a second simulation result corresponding to each deviated pupil condition according to a second principal ray incident angle dot pattern corresponding to each deviated pupil condition and a fourth principal ray incident angle dot pattern;

[0328] According to the first simulation result and the second simulation result corresponding to each pupil deviation condition, a dynamic distortion simulation result of the near-eye display optical system to be simulated is determined.

[0329] In one embodiment, the dynamic distortion simulation module 2604 is also used to

[0330] Determine a third simulation result corresponding to each deviated pupil condition according to a difference between the second simulation result corresponding to each deviated pupil condition and the first simulation result;

[0331] According to the third simulation results corresponding to the multiple pupil deviation conditions, the dynamic distortion simulation result of the near-eye display optical system to be simulated is determined.

[0332] In one embodiment, the model building module is also used to

[0333] For each of the plurality of deviated pupil conditions, determining a corresponding second image plane image height dot map according to a first principal ray incident angle dot map corresponding to each deviated pupil condition;

[0334] Determine a corresponding third image plane image height dot matrix diagram according to the second principal ray incident angle dot matrix diagram corresponding to each pupil deviation condition;

[0335] Generate a text video according to the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition;

[0336] Based on text videos, a dynamic distortion perception model is established.

[0337] In one embodiment, the model building module is also used to

[0338] Get the original text image;

[0339] Mapping the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each pupil deviation condition to determine the overall homography matrix corresponding to each pupil deviation condition;

[0340] According to the overall homography matrix corresponding to each eccentric pupil condition, the original text image is mapped to obtain a mapped text image corresponding to each eccentric pupil condition;

[0341] A text video is generated according to the mapped text images corresponding to multiple pupil deviation conditions.

[0342] In one embodiment, the model building module is also used to

[0343] Performing region division on the second image plane image height dot matrix corresponding to each pupil deviation condition to obtain a first number of first divided sub-regions;

[0344] Performing local mapping on a first number of first divided sub-regions to determine a corresponding plurality of first local homography mapping matrices;

[0345] Performing region division on the third image plane image height dot matrix map corresponding to each pupil deviation condition to obtain a second number of second divided sub-regions; wherein the second number is equal to the first number;

[0346] Performing local mapping on a second number of second divided sub-regions to determine a corresponding plurality of second local homography mapping matrices;

[0347] According to the multiple first local homography mapping matrices and the multiple second local homography mapping matrices corresponding to each deviated pupil condition, the overall homography matrix corresponding to each deviated pupil condition is determined.

[0348] In one embodiment, the model building module is also used to

[0349] Determine the relative position of each first divided sub-region in the image height dot matrix of the second image plane;

[0350] Determine the relative position of each second divided sub-region in the image height dot matrix of the third image plane;

[0351] Performing spatial distortion removal processing on a first local homography mapping matrix corresponding to the first divided sub-region and a second local homography mapping matrix corresponding to the second divided sub-region at the same relative position, to obtain an intermediate local homography mapping matrix corresponding to each relative position;

[0352] According to a plurality of intermediate local homography mapping matrices, a corresponding overall homography matrix is ​​determined.

[0353] In one embodiment, the model building module is also used to

[0354] The dynamic distortion perception model is sent to the first terminal, so that the first terminal outputs the user's perception result of the dynamic distortion of the near-eye display optical system according to the dynamic distortion perception model.

[0355] In one embodiment, the model building module is also used to

[0356] Based on the dynamic distortion simulation results of the near-eye display optical system to be simulated and the target surface defect information, an objective function model is established; the objective function model is used to determine the correlation between dynamic distortion and surface defects;

[0357] The objective function model is sent to the second terminal, so that the second terminal outputs the sensitivity of the near-eye display optical system to surface defects according to the objective function model.

[0358] Each module in the above-mentioned dynamic distortion simulation device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0359] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 27 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store dynamic distortion simulation related data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a dynamic distortion simulation method is implemented.

[0360] Those skilled in the art will understand that Fig. 27 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0361] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0362] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0363] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0364] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0365] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0366] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A dynamic distortion simulation method, characterized in that: The method comprises: Obtaining an object plane dot pattern and optical information corresponding to the near-eye display optical system to be simulated; Based on the optical information, determining a plurality of first image plane chief ray incident angle dot diagrams corresponding to the object plane dot diagram under a plurality of pupil deviation conditions; Based on the optical information, determine a second image plane principal ray incident angle dot pattern corresponding to the object plane dot pattern under an unbiased pupil condition; Based on the plurality of first image plane principal ray incident angle dot matrix diagrams and the second image plane principal ray incident angle dot matrix diagrams, a dynamic distortion simulation result of the near-eye display optical system to be simulated is determined.

2. The method according to claim 1, characterized in that The step of obtaining the object plane dot map comprises: Obtaining a preset image plane principal ray incident angle dot matrix diagram; Performing image height processing on the preset image plane chief ray incident angle dot matrix to determine a corresponding first image plane image height dot matrix; Ray tracing is performed on the first image plane image height dot map to determine the object plane dot map.

3. The method according to claim 2, characterized in that The preset image plane principal ray incident angle dot matrix includes a plurality of principal ray incident angle data; the image height processing is performed on the preset image plane principal ray incident angle dot matrix to determine the corresponding first image plane image height dot matrix, including: Performing image height processing on the plurality of principal ray incident angle data to obtain a plurality of corresponding image height values; According to the plurality of image height values, an image height dot matrix of the first image plane is determined.

4. The method according to claim 1, characterized in that: The optical information includes pupil posture; and determining, based on the optical information, a plurality of first image plane principal ray incident angle dot diagrams corresponding to the object plane dot diagram under a plurality of pupil deviation conditions, comprises: Acquire multiple first pupil rotation angles and eye movement function models; Determine a first pupil deviation corresponding to each first pupil rotation angle based on the eye movement function model and a plurality of the first pupil rotation angles; Determining, according to the plurality of first pupil deviations, a pupil position corresponding to each of the first pupil deviations; each of the pupil positions has a corresponding pupil deviation condition; Based on the optical information and the object plane dot diagram, a first image plane chief ray incident angle dot diagram corresponding to each pupil deviation condition is determined.

5. The method according to claim 4, characterized in that Before obtaining the plurality of first pupil rotation angles and the eye movement function model, the method includes: Obtaining a pupil rotation model and a plurality of second pupil rotation angles; Based on the pupil rotation model and the second pupil rotation angles, determining a corresponding plurality of second pupil deviations; An eye movement function model is established according to the plurality of second pupil rotation angles and the plurality of second pupil deviation amounts.

6. The method according to claim 4, characterized in that The optical information includes first optical information and second optical information; The determining, based on the optical information and the object plane dot map, of the first image plane chief ray incident angle dot map corresponding to each pupil deviation condition comprises: For each of the multiple deviated pupil conditions, based on the first optical information and the object plane dot map, determine a first principal ray incident angle dot map corresponding to each deviated pupil condition; the first optical information refers to optical information corresponding to the near-eye display optical system to be simulated in the absence of target surface defect information; Based on the second optical information and the object plane dot pattern, determining a second principal ray incident angle dot pattern corresponding to each pupil deviation condition; the second optical information refers to optical information corresponding to the near-eye display optical system to be simulated in the presence of target surface defect information; According to the first principal ray incident angle dot pattern and the second principal ray incident angle dot pattern corresponding to each deviated pupil condition, a first image plane principal ray incident angle dot pattern corresponding to each deviated pupil condition is obtained.

7. The method according to claim 6, characterized in that Before determining the second chief ray incident angle dot pattern corresponding to each pupil deviation condition based on the second optical information and the object plane dot pattern, the method further includes: Acquire a preset position and surface defect data; the preset position is located within a preset surface of the near-eye display optical system to be simulated; Converting the surface defect data into a certain format to determine target surface defect information; The target surface defect information is imported into the preset position to determine the second optical information corresponding to the near-eye display optical system to be simulated.

8. The method according to claim 6, characterized in that The optical information includes first optical information and second optical information; and determining, based on the optical information, a second image plane principal ray incident angle dot diagram corresponding to the object plane dot diagram under an unbiased pupil condition, comprises: Determine a third principal ray incident angle dot diagram corresponding to an unbiased pupil condition based on the first optical information and the object plane dot diagram; Determine a fourth principal ray incident angle dot diagram corresponding to an unbiased pupil condition based on the second optical information and the object plane dot diagram; A second image plane chief ray incident angle dot matrix diagram is obtained according to the third chief ray incident angle dot matrix diagram and the fourth chief ray incident angle dot matrix diagram.

9. The method according to claim 8, characterized in that The step of determining the dynamic distortion simulation result of the near-eye display optical system to be simulated based on the plurality of first image plane principal ray incident angle dot patterns and the second image plane principal ray incident angle dot patterns comprises: For each of the plurality of deviated pupil conditions, determine a first simulation result corresponding to each deviated pupil condition according to the first principal ray incident angle dot pattern corresponding to each deviated pupil condition and the third principal ray incident angle dot pattern; Determine a second simulation result corresponding to each deviated pupil condition according to the second principal ray incident angle dot pattern corresponding to each deviated pupil condition and the fourth principal ray incident angle dot pattern; The dynamic distortion simulation result of the near-eye display optical system to be simulated is determined according to the first simulation result and the second simulation result corresponding to each deviated pupil condition.

10. The method according to claim 9, characterized in that Determining the dynamic distortion simulation result of the near-eye display optical system to be simulated according to the first simulation result and the second simulation result corresponding to each pupil deviation condition includes: Determine a third simulation result corresponding to each deviated pupil condition according to a difference between the second simulation result corresponding to each deviated pupil condition and the first simulation result; According to the third simulation results corresponding to the multiple pupil deviation conditions, the dynamic distortion simulation result of the near-eye display optical system to be simulated is determined.

11. The method according to claim 6, characterized in that The method further comprises: For each of the plurality of deviated pupil conditions, determining a corresponding second image plane image height dot map according to the first principal ray incident angle dot map corresponding to each deviated pupil condition; Determine a corresponding third image plane image height dot map according to the second principal ray incident angle dot map corresponding to each pupil deviation condition; Generate a text video according to the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition; Based on the text video, a dynamic distortion perception model is established.

12. The method according to claim 11, characterized in that The step of generating a text video according to the second image plane image height dot matrix map and the third image plane image height dot matrix map corresponding to each pupil deviation condition includes: Get the original text image; Mapping the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each pupil deviation condition to determine the overall homography matrix corresponding to each pupil deviation condition; According to the overall homography matrix corresponding to each skewed pupil condition, mapping processing is performed on the original text image to obtain a mapped text image corresponding to each skewed pupil condition; A text video is generated according to the mapped text images corresponding to a plurality of pupil deviation conditions.

13. The method according to claim 12, characterized in that The mapping of the second image plane image height dot matrix and the third image plane image height dot matrix corresponding to each pupil deviation condition to determine the overall homography matrix corresponding to each pupil deviation condition includes: Performing region division on the second image plane image height dot matrix corresponding to each pupil deviation condition to obtain a first number of first divided sub-regions; Performing local mapping on a first number of the first divided sub-regions to determine a corresponding plurality of first local homography mapping matrices; Performing region division on the third image plane image height dot matrix map corresponding to each pupil deviation condition to obtain a second number of second divided sub-regions; wherein the second number is equal to the first number; Performing local mapping on a second number of the second divided sub-regions to determine a corresponding plurality of second local homography mapping matrices; According to the multiple first local homography mapping matrices and the multiple second local homography mapping matrices corresponding to each deviated pupil condition, the overall homography matrix corresponding to each deviated pupil condition is determined.

14. The method according to claim 13, characterized in that The determining, according to the plurality of the first local homography mapping matrices and the plurality of the second local homography mapping matrices corresponding to each deviated pupil condition, an overall homography matrix corresponding to each deviated pupil condition comprises: Determining the relative position of each of the first divided sub-regions in the image height dot matrix of the second image plane; Determine the relative position of each of the second divided sub-areas in the image height dot matrix of the third image plane; Performing spatial distortion removal processing on a first local homography mapping matrix corresponding to the first divided sub-regions and a second local homography mapping matrix corresponding to the second divided sub-regions having the same relative position, to obtain an intermediate local homography mapping matrix corresponding to each relative position; According to the plurality of intermediate local homography mapping matrices, a corresponding overall homography matrix is ​​determined.

15. The method according to claim 11, characterized in that The method further comprises: The dynamic distortion perception model is sent to the first terminal, so that the first terminal outputs the user's perception result of the dynamic distortion of the near-eye display optical system according to the dynamic distortion perception model.

16. The method according to claim 7, characterized in that The method further comprises: Based on the dynamic distortion simulation results of the near-eye display optical system to be simulated and the target surface defect information, an objective function model is established; the objective function model is used to determine the correlation between the dynamic distortion and the surface defect; The objective function model is sent to the second terminal, so that the second terminal outputs the sensitivity of the near-eye display optical system to surface defects according to the objective function model.

17. A dynamic distortion simulation device, characterized in that: The device comprises: An acquisition module, used to acquire an object plane dot pattern and optical information corresponding to a near-eye display optical system to be simulated; A first determination module is used to determine, based on the optical information, a plurality of first image plane chief ray incident angle dot diagrams corresponding to the object plane dot diagram under a plurality of deviated pupil conditions; A second determination module is used to determine, based on the optical information, a second image plane principal ray incident angle dot pattern corresponding to the object plane dot pattern under an unbiased pupil condition; A dynamic distortion simulation module is used to determine the dynamic distortion simulation result of the near-eye display optical system to be simulated based on a plurality of first image plane principal ray incident angle dot matrix diagrams and second image plane principal ray incident angle dot matrix diagrams.

Images