Model configuration method, device, system and medium based on optical waveguide combiner

Abstract

The application provides a model configuration method, device, system and medium based on an optical waveguide combiner, the method comprising: obtaining a plurality of sample image sets, inputting each sample image set into a corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets; iteratively training a plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models; and configuring a matched target neural network model for at least one optical waveguide combiner based on the corresponding relationship between each target neural network model and each category of optical waveguide combiner, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the matched target neural network model. The above method can improve the display quality of the virtual image output by different optical waveguide combiners.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of extended reality, and particularly relates to a model configuration method, device and system based on an optical waveguide combiner and a medium. BACKGROUND

[0002] Existing near-eye display devices such as augmented reality (AR) and mixed reality (MR) often include an optical waveguide combiner. The function of the optical waveguide combiner is to transmit the image generated by the optical mechanical module into the human eye to present a virtual image. The user can observe the real world scene while seeing the virtual image, and even can interact with it.

[0003] However, the light beam transmission performance of the current optical waveguide combiner is limited by the manufacturing process. The light beams coupled into the optical waveguide combiner at different field angles experience different propagation paths. These factors are difficult to correct when designing the optical waveguide combiner, so there are great differences in the display effect of the virtual image presented by different optical waveguide combiners, which may reduce the display quality of the virtual image. SUMMARY

[0004] The main purpose of the present application is to provide a model configuration method, device, system and medium based on an optical waveguide combiner, and particularly to provide a model configuration method, computer device, model configuration system and storage medium based on an optical waveguide combiner. The present application aims to solve the problem that the display effect of the virtual image presented by different optical waveguide combiners is greatly different, which may reduce the display quality of the virtual image.

[0005] In a first aspect, the present application provides a model configuration method, comprising:

[0006] obtaining a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners;

[0007] inputting each of the sample image sets into the corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets;

[0008] iteratively training a plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models, wherein the plurality of neural network models correspond one-to-one to the plurality of categories of optical waveguide combiners;

[0009] According to the correspondence between each target neural network model and each category of the optical waveguide combiner, a matched target neural network model is configured for each optical waveguide combiner, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the matched target neural network model.

[0010] In a second aspect, the present application also provides a model configuration method based on an optical waveguide combiner, comprising:

[0011] A plurality of sample image sets are obtained, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners;

[0012] Each sample image set is input into a corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets;

[0013] According to the plurality of sample image sets and the plurality of output image sets, a pre-set neural network model is iteratively trained to obtain a trained target neural network model;

[0014] The target neural network model is configured for a plurality of optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the target neural network model.

[0015] In a third aspect, the present application also provides a model configuration method based on an optical waveguide combiner, comprising:

[0016] A plurality of sample image sets are obtained, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners;

[0017] Each sample image set is input into a corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets;

[0018] According to the plurality of sample image sets, the plurality of output image sets, and the optical performance parameters of the plurality of categories of optical waveguide combiners, a pre-set neural network model is iteratively trained to obtain a trained target neural network model;

[0019] The target neural network model is configured for a plurality of optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the matched target neural network model.

[0020] In a fourth aspect, the present application also provides a computer device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection communication between the processor and the memory, wherein the computer program, when executed by the processor, realizes the model configuration method according to any one of the embodiments of the present application.

[0021] In a fifth aspect, the present application also provides a model configuration system, comprising:

[0022] a plurality of near-eye display devices, comprising optical waveguide combiners and storage devices;

[0023] a computer device according to the embodiments of the present application, which is in communication connection with the plurality of near-eye display devices, and is configured to determine a target neural network model matched with the optical waveguide combiner in each of the near-eye display devices, and send the matched target neural network model to the storage device in each of the near-eye display devices, so that the corresponding target neural network model in the storage device is called in advance to correct the virtual image when the optical waveguide combiner transmits the virtual image.

[0024] In a sixth aspect, the present application also provides a storage medium for computer readable storage, characterized in that the storage medium stores one or more programs, which can be executed by one or more processors to realize the model configuration method according to any one of the embodiments of the present application.

[0025] The embodiments of the present application provide a model configuration method based on an optical waveguide combiner, a target neural network model matched with different categories of optical waveguide combiners is trained, and a plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners. Therefore, the target neural network model can be configured for optical waveguide combiners with different optical performance parameters, so that the virtual image can be corrected in advance by the matched target neural network model, and the corrected virtual image is transmitted by the optical waveguide combiner, thereby improving the display quality of the virtual image output by different optical waveguide combiners. Therefore, the problem that the display effect of the virtual image presented by different optical waveguide combiners has a large difference and thus the display quality of the virtual image is reduced can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1 is an application scenario diagram of the optical waveguide combiner provided by the embodiment of the present application;

[0028] Figure 2 is a step flow diagram of the model configuration method based on the optical waveguide combiner provided by the embodiment of the present application;

[0029] Figure 3 is an application scenario diagram of the near-eye display device provided by the embodiment of the present application;

[0030] Figure 4 is an application scenario diagram of the model configuration method provided by the embodiment of the present application;

[0031] Figure 5 is another application scenario diagram of the model configuration method provided by the embodiment of the present application;

[0032] Figure 6 is a scenario diagram of the model configuration method provided by the embodiment of the present application;

[0033] Figure 7 is a step flow diagram of another model configuration method based on the optical waveguide combiner provided by the embodiment of the present application;

[0034] Figure 8 is another scenario diagram of the model configuration method provided by the embodiment of the present application;

[0035] Figure 9 is a step flow diagram of still another model configuration method based on the optical waveguide combiner provided by the embodiment of the present application;

[0036] Figure 10 is still another scenario diagram of the model configuration method provided by the embodiment of the present application;

[0037] Figure 11 is a schematic block diagram of a computer device provided by the embodiment of the present application;

[0038] Figure 12 is a schematic block diagram of a model configuration system provided by the embodiment of the present application.

[0039] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0040] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below; obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of the present application.

[0041] The flowcharts shown in the drawings are only illustrative, and do not necessarily include all the contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0042] The optical waveguide combiner related to the embodiments of the present application is a device capable of coupling the mode and optical energy of the transmitted light wave. The optical waveguide combiner is divided into various types, including directional couplers, grating couplers, Y-branch couplers, multi-mode interference couplers, etc., which work between different optical waveguides. The optical waveguide combiner can include a coupling-in module, a coupling-out module and other components.

[0043] The near-eye display device related to the embodiments of the present application is a glass or goggle type wearable display device composed of a micro display panel and an imaging optical device. The near-eye display device is close to the eye, and the light emitted by the micro display panel is collimated through the imaging optical device to form a virtual image at a far distance where the eye can comfortably focus. The imaging optical device can include an optical waveguide combiner.

[0044] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0045] Figure 1 is an application scenario diagram of the optical waveguide combiner provided by the embodiments of the present application. As shown in Figure 1 The light beam carrying image information is emitted by the light source 10, and then coupled into the optical waveguide combiner 30 through the coupling-in module 20. The light coupled into the optical waveguide combiner 30 is transmitted in the optical waveguide combiner 30 at an angle satisfying the total internal reflection condition, until it is emitted to the human eye at the coupling-out module 40.

[0046] However, the optical waveguide combiner 30 is limited by the actual manufacturing process, and the flatness of its surface will affect the phase, polarization and other parameters of the light transmitted in the optical waveguide combiner 30. Therefore, the light beams of different field angles will experience different propagation paths after being coupled into the optical waveguide combiner 30, which will result in differences in optical performance of each optical waveguide combiner 30. These factors are difficult to correct when designing the optical waveguide combiner 30, so the display effect of the virtual image presented in the human eye will decrease.

[0047] Based on this, the embodiment of the present application provides a model configuration method based on an optical waveguide combiner, a computer device, a model configuration system and a storage medium. The model configuration method based on the optical waveguide combiner can be applied to a computer device, which can include a terminal device or a server. The terminal device can be an electronic device such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant and a wearable device. The wearable device can include a near-eye display device. The server can be a single server or a server cluster composed of multiple servers.

[0048] Please refer to Figure 2 , Figure 2 The steps of the model configuration method based on the optical waveguide combiner provided by the embodiment of the present application are shown in a flowchart. The model configuration method includes:

[0049] S101, a plurality of sample image sets are obtained, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners.

[0050] The sample image set includes a plurality of sample images, which are a training sample set of a neural network model. For optical waveguide combiners of different categories, a corresponding sample image set can be established. Each category of optical waveguide combiner corresponds to a plurality of sample image sets, and the sample image sets corresponding to optical waveguide combiners of different categories can be the same or different.

[0051] Because the optical performance parameters of different optical waveguide combiners are different, the optical transmission performance between different optical waveguide combiners can have large differences, so that the display effect of the virtual image presented after transmission through different optical waveguide combiners has large differences.

[0052] Therefore, the optical waveguide combiner can be divided into multiple categories, and the multiple categories of optical waveguide combiners can be determined according to the optical performance parameters of the multiple optical waveguide combiners. The optical transmission performance of the optical waveguide combiner can be determined by the optical performance parameters of the optical waveguide combiner. The optical transmission performance of the optical waveguide combiner of the same category is similar, and the optical transmission performance of the optical waveguide combiner of different categories is different.

[0053] In an embodiment, the optical performance parameters can include parameters such as modulation transfer function (MTF), ghosting, diffraction efficiency, uniformity, etc. The optical performance parameters can also include parameters such as the aperture function Q of the input grating, the waveguide combiner transfer function H, and the aperture function P of the output grating. Therefore, the optical waveguide combiners with large differences in optical transmission performance can be divided into multiple categories based on the optical performance parameters of the optical waveguide combiners.

[0054] Exemplarily, optical performance parameters of the plurality of optical waveguide combiners are acquired; the plurality of optical waveguide combiners are evaluated according to the optical performance parameters of the plurality of optical waveguide combiners, to obtain evaluation results of the plurality of optical waveguide combiners; and the plurality of optical waveguide combiners are classified based on the evaluation results of the plurality of optical waveguide combiners, to obtain a plurality of categories of the optical waveguide combiners.

[0055] The evaluation of the plurality of optical waveguide combiners can be a rating or a score, and the evaluation result can be a rating result or a score result. The specific evaluation method can be flexibly set according to actual conditions. For example, the optical performance parameters include MTF, ghosting, diffraction efficiency, and uniformity. The score value of the plurality of optical waveguide combiners is determined based on the numerical values of the MTF, the ghosting, the diffraction efficiency, and the uniformity. The plurality of optical waveguide combiners are divided into a plurality of different categories A, B, C, and D according to the score value of each optical waveguide combiner. The score values of the optical waveguide combiners in the same category are in the same set score segment, and the score values of the optical waveguide combiners in different categories are in different set score segments.

[0056] Exemplarily, optical performance parameters of the plurality of optical waveguide combiners are acquired; the plurality of optical waveguide combiners are evaluated according to the optical performance parameters of the plurality of optical waveguide combiners, to obtain evaluation results of the plurality of optical waveguide combiners; and the plurality of optical waveguide combiners are classified based on the evaluation results of the plurality of optical waveguide combiners, to obtain a plurality of categories of the optical waveguide combiners.

[0057] The optical performance parameters include an aperture function Q of an input grating and a waveguide combiner transfer function H. The score value or the rating of the plurality of optical waveguide combiners can be determined based on the optical performance parameters. The plurality of optical waveguide combiners can be divided into a plurality of different categories A, B, C, and D according to the score value or the rating of each optical waveguide combiner.

[0058] S102, each sample image set is input into a corresponding category of optical waveguide combiner for processing, to obtain a plurality of output image sets.

[0059] It should be noted that each sample image set needs to be processed based on an actually manufactured optical waveguide combiner, to obtain a corresponding output image set.

[0060] In an embodiment, a near-eye display device includes an image source module, an optical waveguide combiner, and a detection module. A plurality of sample images in a sample image set are sequentially loaded into light signals of the image source module. The image source module outputs the light signals loaded with the sample images to the optical waveguide combiner, and transmits the light signals to the detection module through the optical waveguide combiner. The output signals of the optical waveguide combiner are collected by the detection module, and the output images in the output signals are extracted, to obtain an output image set.

[0061] Exemplarily, as shown in FIG. 1, an optical waveguide combiner 100 is provided. The optical waveguide combiner 100 includes an input grating 101, a waveguide 102, and an output grating 103. Figure 3As shown, the near-eye display device includes an image source module 31, an optical waveguide combiner 30 and a detection module 32. The input signal (sample image) is sequentially loaded onto the light signal on the image source module 31, and then the light signal carrying the sample image information is transmitted to the detection module 40 through the optical waveguide combiner 30, and the output signal of the optical waveguide combiner 30 is collected through the detection module 40 and the output image is extracted therefrom to obtain an output image set.

[0062] S103, according to the plurality of sample image sets and the plurality of output image sets, iteratively training the plurality of neural network models to obtain a plurality of trained target neural network models.

[0063] It should be noted that the output image set is obtained based on inputting the sample image set into the corresponding category of the optical waveguide combiner for processing, so that the neural network model is used to simulate the optical waveguide combiner, and the neural network model is iteratively trained based on the sample image set and the output image set. The trained target neural network model can have the optical transmission characteristics of the corresponding optical waveguide combiner.

[0064] It should be noted that after obtaining the plurality of sample image sets and the plurality of output image sets, the neural network model can be pre-established based on the plurality of categories of the optical waveguide combiner, so that the plurality of sample image sets and the plurality of output image sets can be directly used to iteratively train the plurality of neural network models. The neural network model can include a convolutional neural network CNN, a recurrent neural network RNN, a deep neural network DNN, etc., or other types of neural network models such as a forward differentiable model, and the present application does not make specific limitations.

[0065] In an embodiment, before iteratively training the plurality of neural network models, a plurality of neural network models corresponding one-to-one to the plurality of categories of the optical waveguide combiner need to be established. Specifically, before iteratively training the plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models, it further includes: determining first configuration parameters of each neural network model according to the optical performance parameters of each category of optical waveguide combiner; determining second configuration parameters of each neural network model according to the image size of each sample image set or each output image set; and constructing the plurality of neural network models based on the first configuration parameters and the second configuration parameters.

[0066] The optical performance parameters of the optical waveguide combiner include an aperture function of an input grating, a waveguide combiner transfer function, and an aperture function of an output grating, and can also include other parameters. The optical performance parameters can be used as hyperparameters of the neural network model, and do not need to participate in training. Therefore, the first configuration parameter can include the hyperparameters of the neural network model. The larger the image size of the sample image set or the output image set, the more levels of the neural network model, such as more hidden layers. Therefore, the second configuration parameter can be a scale parameter used to represent the neural network model. Based on the first configuration parameter and the second configuration parameter, the neural network model corresponding to each category of the optical waveguide combiner can be constructed, thereby improving the matching degree between the neural network model and the corresponding category of the optical waveguide combiner, and improving the reliability of the neural network model in actual application.

[0067] In an embodiment, the training process of the neural network model includes: inputting a first sample image in the sample image set into the corresponding neural network model for processing to obtain a first output image; calculating a first error value of the first output image and a corresponding output image in the output image set; when the first error value is greater than a preset error value, updating the model parameters of the neural network model according to the first error value, and iteratively training the neural network model with the updated model parameters; and when the first error value is less than or equal to the preset error value, determining the neural network model with the updated model parameters as a trained target neural network model.

[0068] The first sample image can be any sample image in the sample image set. The first sample image is input into the neural network model as an input, and the first output image is obtained after processing by the neural network model. Since the model parameters of the neural network model are random at the beginning, there is a difference between the prediction result (the first output image) of the neural network model and the true result (the corresponding output image in the output image set) obtained by processing the optical waveguide combiner of the corresponding category.

[0069] It should be noted that the first error value is the difference value between the prediction result of the neural network model and the true result, and the first error value is, for example, a peak signal-to-noise ratio. The model parameters of the neural network model can be updated based on the back propagation algorithm according to the first error value. Iterative training of the neural network model with the updated model parameters can be to continue the step of inputting the first sample image in the sample image set into the corresponding neural network model for processing to obtain the first output image. After training of N groups of images, the difference value between the prediction result of the neural network model and the true result is less than the preset error value. At this time, the model parameter training is completed, and a trained target neural network model is obtained.

[0070] For example, Figure 4As shown, the first sample image 41 is input into the neural network model, and the first output image 42 is obtained at the output end of the neural network model. The first error value 44 of the first output image 42 and the corresponding output image 43 in the output image set is calculated, and the model parameters are updated according to the first error value 44 and the back propagation algorithm. After the iterative training of the neural network model by the N groups of first sample images 41, the first error value 44 is less than the preset error value, and at this time the model parameter training is completed. Finally, the prediction accuracy of the neural network model can also be tested by using test image data.

[0071] In another embodiment, whether the trained target neural network model is obtained can also be determined according to the number of iterations of the neural network model, the length of the iterative training, and the like, which are not limited in the present application. For example, when the number of iterations is greater than the preset number of training, or when the length of the iterative training is greater than the preset length, the currently trained neural network model is determined as the trained target neural network model.

[0072] In an embodiment, the model configuration method further comprises: inputting a second sample image in the sample image set into the trained target neural network model for processing to obtain a second output image; calculating a second error value of the second output image and a corresponding expected output image; and updating the second sample image based on the second error value using the back propagation algorithm, so that the second error value of the updated second sample image and the expected output image is less than or equal to a preset error value.

[0073] The second sample image can be any sample image in the sample image set, and the expected output image can be the best display image of the second sample image. The expected output image can be set and have the best display effect. The second error value can be a peak signal-to-noise ratio parameter. Based on the second error value and the back propagation algorithm, the second sample image is updated, which can realize the optimization of the second sample image, so that the updated second sample image is closer to the output image of the real optical waveguide combiner after being transmitted by the optical waveguide combiner.

[0074] It should be noted that based on the trained target neural network model, the sample images in the sample image set can be optimized, and the display effect of the virtual image presented by the optimized sample images after being transmitted by different optical waveguide combiners is the best.

[0075] For example, the sample image set comprises a plurality of sample images, and the model configuration method comprises: inputting the sample image set into the neural network model for iterative training to obtain a trained target neural network model. Figure 5As shown, on the basis of the trained target neural network model, a second sample image 51 is randomly input, and the output result obtained through the target neural network model is a second output image 52, which is consistent with the output image output by the optical waveguide combiner. This is because, after training of the multiple sample image sets, the target neural network model has all the characteristics of the optical waveguide combiner. At this time, the second output image 52 is consistent with the output image output by the optical waveguide combiner, and the second output image 52 also has the problem of display effect degradation. Then, the second error value 54 between the output result, i.e., the second output image 52, and the expected output image 53 is calculated, and the second sample image 51 is updated according to the back propagation algorithm until the second error value 54 of the second output image 52 and the expected output image 53 is less than or equal to the preset error value. At this time, the updated sample image 55 obtained is an optimized image that can be used for the optical waveguide combiner.

[0076] In S104, based on the correspondence between each target neural network model and each category of optical waveguide combiner, a target neural network model matched with each optical waveguide combiner is configured, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the matched target neural network model.

[0077] It should be noted that, based on the foregoing step operation, the correspondence between each target neural network model and each category of optical waveguide combiner can be obtained, and based on the correspondence, a target neural network model matched with each optical waveguide combiner can be configured for the multiple optical waveguide combiners. Configuring a target neural network model matched with each optical waveguide combiner can mean that each target neural network model is respectively stored in a storage device corresponding to each optical waveguide combiner, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by calling the target neural network model in the corresponding storage device. The storage device can be integrated with the near-eye display device together with the optical waveguide combiner, or the storage device can be a cloud server.

[0078] It should be noted that, in actual use of a certain type of optical waveguide combiner, the input image needs to be corrected and optimized according to the target neural network model configured for it, and then transmitted to the human eye through the optical waveguide combiner, so as to ensure that the optical waveguide combiners of different categories can participate in the correction of the virtual image in advance through the matched target neural network model, so as to ensure that the optical waveguide combiners of different categories can have their own best display effect. For example, in the output process of the virtual image, the virtual image is first corrected by the matched target neural network model, and then the corrected virtual image is transmitted by the optical waveguide combiner.

[0079] An exemplary near-eye display device includes an optical waveguide combiner and a storage device. Based on a correspondence between each target neural network model and each category of the optical waveguide combiner, a matching target neural network model is configured for each optical waveguide combiner; a matching target neural network model of the optical waveguide combiner in each near-eye display device is determined, and the matching target neural network model is sent to the storage device in each near-eye display device, so that when each optical waveguide combiner transmits a virtual image, the target neural network model in the corresponding storage device is called in advance to correct the virtual image.

[0080] In an embodiment, a mapping relationship table is obtained, which records a correspondence between categories of a plurality of optical waveguide combiners and a plurality of trained target neural network models; based on the mapping relationship table, a matching target neural network model of each optical waveguide combiner is determined; and each target neural network model is stored in a corresponding storage device of each optical waveguide combiner.

[0081] The correspondence between the categories of the plurality of optical waveguide combiners and the plurality of trained target neural network models is recorded to obtain a mapping relationship table. The mapping relationship table can be stored in the memory of the computer device. When a certain type of optical waveguide combiner is needed, a target neural network model matching the optical waveguide combiner of that type can be determined based on the mapping relationship table, and each target neural network model is stored in a corresponding storage device of the optical waveguide combiner, so that the input image can be corrected using the target neural network model before being transmitted through the optical waveguide combiner to obtain a corresponding virtual image. The virtual image is transmitted to the human eye, and the update mode of the input image can refer to the related examples of the second sample image in the above embodiments, which are not limited in the present application.

[0082] Please refer to Figure 6 , Figure 6 A scene schematic diagram of the model configuration method provided by the embodiments of the present application.

[0083] As Figure 6 shown, the neural network model includes model 1 to model 9, which correspond to category 1 to category 9 of the optical waveguide combiner, respectively. When a certain type of optical waveguide combiner is needed, a matching target neural network model can be determined from model 1 to model 9 based on the category of the optical waveguide combiner. When the input image of the optical waveguide combiner is input image set 1 to input image set 9, the input image in input image set 1 to input image set 9 can be updated in turn using the target neural network model, and then the corresponding virtual image is obtained by transmitting through the optical waveguide combiner and projected to the human eye, so that the optical waveguide combiner of different categories can have their own best display effect, greatly improving the display quality of the virtual image.

[0084] The model configuration method based on the optical waveguide combiner provided by the above embodiment trains a target neural network model matched for different categories of optical waveguide combiners, so that the target neural network model can be configured for optical waveguide combiners with different optical transmission performances to output a virtual image, and the optimal display effect of the virtual image can be achieved for different optical waveguide combiners. Therefore, the problem that the display effect of the virtual image presented through different optical waveguide combiners has a large difference, and thus the display quality of the virtual image is reduced, can be solved.

[0085] Please refer to Figure 7 , Figure 7 The step flowchart of another model configuration method based on the optical waveguide combiner provided by the embodiment of the present application is shown.

[0086] As shown in Figure 7 , the model configuration method includes steps S201 to S204.

[0087] Step S201, a plurality of sample image sets are obtained, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners.

[0088] The sample image set includes a plurality of sample images, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners. For optical waveguide combiners of different categories, a corresponding sample image set can be established, and each category of optical waveguide combiner corresponds to at least one sample image set. The sample image sets corresponding to optical waveguide combiners of different categories can be the same or different.

[0089] The optical performance parameters can include modulation transfer function, ghosting, diffraction efficiency, uniformity, etc. The optical performance parameters can also include aperture function Q of the input grating, waveguide combiner transfer function H, aperture function P of the output grating, etc.

[0090] Step S202, each sample image set is input into the corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets.

[0091] Each sample image set needs to be processed based on an actually manufactured optical waveguide combiner to obtain the corresponding output image set. Therefore, based on the correspondence between the sample image set and the category of the optical waveguide combiner, the plurality of sample images in the sample image set are sequentially input into the corresponding category of optical waveguide combiner for processing to obtain the output image set of the sample image set.

[0092] Step S203, the pre-set neural network model is iteratively trained according to the plurality of sample image sets and the plurality of output image sets to obtain a trained target neural network model.

[0093] The pre-set neural network model can be a large model. The large model can be used to simulate multiple categories of optical waveguide combiners. Based on the multiple sample image sets and the multiple output image sets, the pre-set neural network model is iteratively trained, and the trained target neural network model can have optical transmission characteristics of multiple categories of optical waveguide combiners.

[0094] In an embodiment, the pre-set neural network model is constructed in the following manner: according to the optical performance parameters of the optical waveguide combiners of each category, a first configuration parameter of the neural network model is determined; according to the image sizes of each sample image set or each output image set, a second configuration parameter of the neural network model is determined; and based on each first configuration parameter and each second configuration parameter, the neural network model is constructed.

[0095] The optical performance parameters of the optical waveguide combiner include the aperture function of the input grating, the waveguide combiner transmission function, and the aperture function of the output grating, and can also include other parameters. The optical performance parameters can be used as hyperparameters of the neural network model and do not need to participate in training, so the first configuration parameter can be an average value or a weighted average value of the optical performance parameters. The larger the image size of the sample image set or the output image set, the more layers of the neural network model, such as more hidden layers, so the second configuration parameter can be a scale parameter for representing the neural network model, and the second configuration parameter can be determined according to the maximum image size of the sample image set or the output image set.

[0096] It should be noted that based on the first configuration parameter and the second configuration parameter, a neural network model that matches multiple categories of optical waveguide combiners can be constructed, thereby improving the matching degree between the neural network model and optical waveguide combiners of different categories and improving the reliability of the neural network model in actual application.

[0097] It can be understood that the training process of the neural network model can refer to the specific training process of the neural network model in the above embodiments, and the embodiments of the present application will not be repeated here.

[0098] Step S204, configuring the target neural network model for the multiple optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the target neural network model.

[0099] The target neural network model of the large model is trained for different categories of optical waveguide combiners, and the input images of different optical waveguide combiners can be pre-corrected based on the target neural network model of the large model, so that the display effect of different categories of optical waveguide combiners can be guaranteed to be good, thereby improving the display quality of the virtual image. Therefore, the problem that the display effect of the virtual image presented by different optical waveguide combiners is greatly different, thereby possibly reducing the display quality of the virtual image, can be solved.

[0100] In an embodiment, the target neural network model is stored in the storage device corresponding to the optical waveguide combiner. When a certain type of optical waveguide combiner needs to be applied, the input image can be pre-corrected using the target neural network model in the storage device, and then transmitted through the optical waveguide combiner to obtain the corresponding virtual image, so that different types of optical waveguide combiners can have good display effects.

[0101] For example, the near-eye display device includes an optical waveguide combiner and a storage device. The target neural network model is configured for different categories of optical waveguide combiners; the target neural network model is sent to the storage device in each near-eye display device, so that when the optical waveguide combiner transmits the virtual image, the target neural network model in the corresponding storage device is called to correct the virtual image.

[0102] Please refer to Figure 8 , Figure 8 Another scene diagram of the model configuration method provided by the embodiment of the present application is provided.

[0103] As shown in Figure 8 , when a certain type of optical waveguide combiner needs to be applied, if the input image of the optical waveguide combiner is input image set 1 to input image set 9, the input image in input image set 1 to input image set 9 can be corrected based on the large model (target neural network model) in turn, and then transmitted through the optical waveguide combiner to obtain the corresponding virtual image and projected to the human eye, so that different categories of optical waveguide combiners can have good display effects, thereby improving the display quality of the virtual image.

[0104] Please refer to Figure 9 , Figure 9 Another step flow diagram of the model configuration method based on the optical waveguide combiner provided by the embodiment of the present application is provided.

[0105] As shown in Figure 9 , the model configuration method includes steps S301 to S304.

[0106] Step S301, obtaining a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners.

[0107] The sample image set includes a plurality of sample images, and the plurality of categories of the optical waveguide combiner are determined according to the optical performance parameters of the plurality of optical waveguide combiners. For different categories of the optical waveguide combiner, a corresponding sample image set can be established, each category of the optical waveguide combiner corresponds to at least one sample image set, and the sample image sets corresponding to different categories of the optical waveguide combiner can be the same or different.

[0108] In step S302, each sample image set is input into the corresponding category of the optical waveguide combiner for processing to obtain a plurality of output image sets.

[0109] Each sample image set needs to be processed based on the actually manufactured optical waveguide combiner to obtain the corresponding output image set. Therefore, based on the correspondence between the sample image set and the category of the optical waveguide combiner, the plurality of sample images in the sample image set are sequentially input into the corresponding category of the optical waveguide combiner for processing to obtain the output image set of the sample image set.

[0110] In step S303, the pre-set neural network model is iteratively trained according to the plurality of sample image sets, the plurality of output image sets, and the optical performance parameters of the plurality of categories of the optical waveguide combiner to obtain a trained target neural network model.

[0111] The pre-set neural network model can be a large model. The large model can be used to simulate the plurality of categories of the optical waveguide combiner. In training the large model (pre-set neural network model), the optical performance parameters of the plurality of categories of the optical waveguide combiner are separately input into the network as hyperparameters of the model for training. After training, the large model (target neural network) can be corrected by actually measuring the optical performance parameters of the optical waveguide combiner when used, so that the output result is more accurate.

[0112] It should be noted that the pre-set neural network model is iteratively trained based on the plurality of sample image sets, the plurality of output image sets, and the plurality of optical performance parameters. The trained target neural network model can have the optical transmission characteristics of the plurality of categories of the optical waveguide combiner, and the output result is more accurate.

[0113] In an embodiment, the pre-set neural network model is constructed in the following manner: determining first configuration parameters of the neural network model according to the optical performance parameters of each category of the optical waveguide combiner; determining second configuration parameters of the neural network model according to the image sizes of each sample image set or each output image set; and constructing the neural network model based on the first configuration parameters and the second configuration parameters.

[0114] The optical performance parameters of the optical waveguide combiner include MTF, ghosting, diffraction efficiency, uniformity, and the like, and can also include other parameters. The optical performance parameters can be used as hyperparameters of the neural network model, and do not need to participate in training, so the first configuration parameter can be an average value or a weighted average value of the optical performance parameters. The larger the image size of the sample image set or the output image set, the more levels of the neural network model, such as more hidden layers, so the second configuration parameter can be a scale parameter for representing the neural network model. The second configuration parameter can be determined according to the maximum image size of the sample image set or the output image set.

[0115] It should be noted that based on the first configuration parameter and the second configuration parameter, a neural network model matched with the multiple categories of optical waveguide combiners can be constructed, thereby improving the matching degree between the neural network model and optical waveguide combiners of different categories, and improving the reliability of the neural network model in actual application.

[0116] In an embodiment, a third sample image in the sample image set and the optical performance parameter of the optical waveguide combiner of the corresponding category are input to a pre-set neural network model for processing to obtain a third output image; a third error value of the third output image and the corresponding output image in the output image set is calculated; the model parameter of the neural network model is updated according to the third error value until the neural network model with the updated model parameter converges, and a trained target neural network model is obtained.

[0117] The third sample image can be any sample image in the sample image set, and the third sample image and the optical performance parameter of the corresponding optical waveguide combiner are collectively used as the input of the neural network model, and the third output image is obtained after processing by the neural network model. Since the model parameters of the neural network model are random at the beginning, there is a difference between the prediction result (the third output image) of the neural network model and the true result (the corresponding output image in the output image set) obtained by processing the optical waveguide combiner of the corresponding category.

[0118] It should be noted that the third error value is the difference value between the prediction result of the neural network model and the true result, such as the peak signal-to-noise ratio, and the model parameters of the neural network model can be updated based on the back propagation algorithm according to the third error value. After training of the neural network model, the difference value between the prediction result and the true result of the neural network model is less than a preset error value, at which time the model parameter training is completed, and a trained target neural network model is obtained.

[0119] It can be understood that the convergence condition of the network model can be that the third error value is less than or equal to the preset error value, or the number of iteration training is greater than the preset number of training, or the iteration training time is greater than the preset time, etc. For example, when the number of iteration training is greater than the preset number of training, or the iteration training time is greater than the preset time, it is determined that the neural network model converges.

[0120] Step S304, configuring a target neural network model for each light waveguide combiner, so that when each light waveguide combiner transmits a virtual image, the virtual image is corrected in advance by a matched target neural network model.

[0121] Among them, a large model of the target neural network model is trained for light waveguide combiners of different categories, and the input image of each light waveguide combiner can be corrected in advance based on the large model of the target neural network model, so that after the corrected virtual image is transmitted by light waveguide combiners of different categories, they can all have good display effects, thereby improving the display quality of the virtual image. Therefore, it can solve the problem that the display effect of the virtual image presented by different light waveguide combiners is greatly different, which may reduce the display quality of the virtual image.

[0122] In an embodiment, the target neural network model is stored in the storage device corresponding to each light waveguide combiner. When a certain type of light waveguide combiner is needed, the input image can be corrected using the target neural network model in the storage device, and then transmitted through the light waveguide combiner to obtain the corresponding virtual image, which can ensure that light waveguide combiners of different categories can have good display effects.

[0123] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described model configuration method can refer to the corresponding process in the foregoing model configuration method embodiments, which will not be described here.

[0124] Please refer to Figure 10 , Figure 10 Another scene diagram of the model configuration method provided by the embodiment of the present application is provided.

[0125] As Figure 10As shown, when a certain type of optical waveguide combiner needs to be applied, if the input image of the optical waveguide combiner is input image set 1 to input image set 9, the input parameters (optical performance parameters) include MTF, ghosting, diffraction efficiency, uniformity, etc. The input images in the input image set 1 to the input image set 9 can be updated in sequence based on the large model (target neural network model), and then the corresponding virtual images are obtained through the transmission of the optical waveguide combiner and projected to the human eye, so that different types of optical waveguide combiners can have good display effects, thereby improving the display quality of the virtual image.

[0126] Referring to Figure 11 , Figure 11 a schematic block diagram of a computer device provided in an embodiment of the present application.

[0127] As Figure 11 shown, the computer device 400 includes a processor 410 and a memory 420, which are connected through a bus 430, such as an I2C (Inter-integrated Circuit) bus.

[0128] Specifically, the processor 410 is configured to provide computing and control capabilities to support the operation of the entire computer device. The processor 410 can be a central processing unit (CPU). The processor 410 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0129] Specifically, the memory 420 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0130] Those skilled in the art can understand Figure 11 that the structure shown in the figure is only a block diagram of part of the structure related to the embodiment of the present application, and does not constitute a limitation on the computer device to which the embodiment of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0131] The processor is configured to run a computer program stored in the memory and implement any one of the model configuration methods provided in the embodiments of the present application when the computer program is executed.

[0132] In an embodiment, the processor is configured to run a computer program stored in the memory and implement the following steps when the computer program is executed: obtaining a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners; inputting each of the sample image sets into an optical waveguide combiner of a corresponding category for processing to obtain a plurality of output image sets; iteratively training a plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models, wherein the plurality of neural network models correspond one-to-one to the plurality of categories of optical waveguide combiners; and configuring a plurality of optical waveguide combiners with matching target neural network models based on a correspondence between each of the target neural network models and each of the categories of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is corrected in advance by the matching target neural network model.

[0133] In an embodiment, the processor is configured to run a computer program stored in the memory and implement the following steps when the computer program is executed: obtaining a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners; inputting each of the sample image sets into an optical waveguide combiner of a corresponding category for processing to obtain a plurality of output image sets; iteratively training a plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models, wherein the plurality of neural network models correspond one-to-one to the plurality of categories of optical waveguide combiners; and configuring a plurality of optical waveguide combiners with matching target neural network models based on a correspondence between each of the target neural network models and each of the categories of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is corrected in advance by the matching target neural network model.

[0134] In an embodiment, the processor is configured to run a computer program stored in the memory and implement the following steps when executing the computer program: obtaining a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners; inputting each of the sample image sets into an optical waveguide combiner of a corresponding category for processing to obtain a plurality of output image sets; performing iterative training on a pre-set neural network model according to the plurality of sample image sets, the plurality of output image sets, and the optical performance parameters of the optical waveguide combiners of the plurality of categories to obtain a trained target neural network model; and configuring the target neural network model for the plurality of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is corrected in advance by the target neural network model.

[0135] It should be noted that, for the convenience and brevity of description, the specific working process of the computer device 400 described above can refer to the corresponding process in the foregoing model configuration method embodiment, which will not be described here.

[0136] Please refer to Figure 12 , Figure 12 The model configuration system provided in the embodiment of the present application is shown in the structural schematic block diagram.

[0137] As Figure 12 shown, the model configuration system 500 includes a plurality of near-eye display devices 510 and a computer device 520.

[0138] The near-eye display device 510 includes an optical waveguide combiner and a storage device. The computer device 520 is in communication connection with each near-eye display device 510. The computer device 520 is configured to determine a target neural network model matched with the optical waveguide combiner in each near-eye display device 510, and send the matched target neural network model to the storage device in each near-eye display device 510, so that when each optical waveguide combiner transmits a virtual image, the corresponding target neural network model in the storage device is called in advance to correct the virtual image.

[0139] In some embodiments, the near-eye display device 510 can include an augmented reality (AR) display device, a mixed reality (MR) display device, etc. The computer device 520 can be the computer device 400 in the foregoing embodiments.

[0140] It should be noted that, for the convenience and brevity of description, the specific working process of the model configuration system 500 described above can refer to the corresponding process in the foregoing model configuration method embodiment, which will not be described here.

[0141] The embodiment of the present application further provides a storage medium for computer readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement any one of the model configuration methods based on the optical waveguide combiner provided by the embodiment of the present application.

[0142] The storage medium can be an internal storage unit of the computer device, for example, a hard disk or a memory of the computer device. The storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate but not limit the technical solutions of the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A model configuration method based on an optical waveguide combiner, characterized by, The method comprises: obtaining a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners; inputting each of the sample image sets into a corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets; iteratively training a plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models, wherein the plurality of neural network models correspond one-to-one to the plurality of categories of optical waveguide combiners; based on the correspondence between each of the target neural network models and each category of the optical waveguide combiners, configuring a matching target neural network model for each of the plurality of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is corrected in advance by the matching target neural network model; wherein the iteratively training the plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models comprises: inputting a first sample image in the sample image set into a corresponding neural network model for processing to obtain a first output image; calculating a first error value of the first output image and a corresponding output image in the output image set; when the first error value is greater than a preset error value, updating model parameters of the neural network model according to the first error value, and iteratively training the neural network model with updated model parameters; when the first error value is less than or equal to the preset error value, determining the neural network model with updated model parameters as a trained target neural network model; before the iteratively training the plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models, further comprising: determining first configuration parameters of each of the neural network models according to optical performance parameters of the optical waveguide combiner of each category; determining second configuration parameters of each of the neural network models according to image sizes of each of the sample image sets or each of the output image sets; and constructing the plurality of neural network models based on the first configuration parameters and the second configuration parameters.

2. The model configuration method of claim 1, wherein, The method further comprises: inputting a second sample image in the sample image set into a trained target neural network model for processing to obtain a second output image; calculating a second error value of the second output image and a corresponding expected output image; based on the second error value, updating the second sample image using a back propagation algorithm, so that the second error value of the updated second sample image and the expected output image is less than or equal to the preset error value.

3. The model configuration method of claim 1, wherein, The optical performance parameters of the optical waveguide combiner include an aperture function of an input grating, a waveguide combiner transfer function, and an aperture function of an output grating.

4. The model configuration method according to any one of claims 1 to 3, characterized by, The corresponding relationship between each target neural network model and the category information of each optical waveguide combiner is used to configure a matching target neural network model for each optical waveguide combiner, including: A mapping relationship table is generated, which records the correspondence between the categories of the plurality of optical waveguide combiners and the plurality of trained target neural network models; Based on the mapping relationship table, the target neural network model matched for each optical waveguide combiner is determined; Each target neural network model is stored in the storage device corresponding to each optical waveguide combiner.

5. The model configuration method according to any one of claims 1 to 3, characterized by, The method further includes: According to the optical performance parameters of the plurality of optical waveguide combiners, the plurality of optical waveguide combiners are evaluated to obtain evaluation results of the plurality of optical waveguide combiners; Based on the evaluation results of the plurality of optical waveguide combiners, the plurality of optical waveguide combiners are classified to obtain a plurality of categories of the optical waveguide combiners.

6. A model configuration method of an optical waveguide combiner, comprising: It includes: A plurality of sample image sets are obtained, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners; Each sample image set is input into a corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets; According to the plurality of sample image sets and the plurality of output image sets, a pre-set neural network model is iteratively trained to obtain a trained target neural network model; The target neural network model is configured for a plurality of optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the target neural network model; Wherein, according to the plurality of sample image sets and the plurality of output image sets, the pre-set neural network model is iteratively trained to obtain a trained target neural network model, including: A first sample image in the sample image set is input into a corresponding neural network model for processing to obtain a first output image; a first error value of the first output image and a corresponding output image in the output image set is calculated; when the first error value is greater than a preset error value, the model parameters of the neural network model are updated according to the first error value, and the neural network model with updated model parameters is iteratively trained; when the first error value is less than or equal to the preset error value, the neural network model with updated model parameters is determined as the trained target neural network model; Before the pre-set neural network model is iteratively trained according to the plurality of sample image sets and the plurality of output image sets to obtain a trained target neural network model, it further includes: According to the optical performance parameters of the category of optical waveguide combiner, the first configuration parameters of the neural network model are determined; according to the image size of each sample image set or each output image set, the second configuration parameters of the neural network model are determined; based on the first configuration parameters and the second configuration parameters, a pre-set neural network model is constructed.

7. A model configuration method of an optical waveguide combiner, comprising: It includes: acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond to a plurality of categories of optical waveguide combiners one by one, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners; input each of the sample image sets into a corresponding category of optical waveguide combiner for processing to obtain a plurality of output image sets; perform iterative training on a pre-set neural network model according to the plurality of sample image sets, the plurality of output image sets, and the optical performance parameters of the optical waveguide combiners of the plurality of categories to obtain a trained target neural network model; configure the target neural network model for the plurality of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is corrected in advance by the pre-matched target neural network model; wherein the performing iterative training on the pre-set neural network model according to the plurality of sample image sets, the plurality of output image sets, and the optical performance parameters of the optical waveguide combiners of the plurality of categories to obtain the trained target neural network model comprises: inputting a third sample image in the sample image set and the optical performance parameters of the optical waveguide combiner of the corresponding category into the pre-set neural network model for processing to obtain a third output image; calculating a third error value of the third output image and a corresponding output image in the output image set; updating model parameters of the neural network model according to the third error value until the neural network model with updated model parameters converges to obtain the trained target neural network model; before the performing iterative training on the pre-set neural network model according to the plurality of sample image sets and the plurality of output image sets to obtain the trained target neural network model, further comprising: determining first configuration parameters of the neural network model according to the optical performance parameters of the optical waveguide combiner of the category; determining second configuration parameters of the neural network model according to image sizes of each of the sample image sets or each of the output image sets; and constructing the pre-set neural network model based on the first configuration parameters and the second configuration parameters.

8. A computer device, comprising: The computer device comprises a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for connecting and communicating between the processor and the memory, wherein the computer program is executed by the processor to implement the model configuration method of any one of claims 1 to 7.

9. A model configuration system characterized by comprising: The model configuration system comprises: a plurality of near-eye display devices, the near-eye display device comprising an optical waveguide combiner and a storage device; The computer device of claim 8 is in communication connection with each of the near-eye display devices, and is used to determine the target neural network model matched with the optical waveguide combiner in each of the near-eye display devices, and send the matched target neural network model to the storage device in each of the near-eye display devices, so that when each of the optical waveguide combiners transmits a virtual image, the corresponding target neural network model in the storage device is called in advance to correct the virtual image.

10. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the model configuration method in any one of claims 1 to 7.

Images