Model configuration method, device and system based on optical waveguide combiner and medium

Abstract

The invention provides a model configuration method, device and system based on an optical waveguide combiner and a medium, and the method comprises the steps: obtaining a plurality of sample image sets, inputting each sample image set into an optical waveguide combiner of a corresponding type for processing, and obtaining a plurality of output image sets; performing iterative training on the plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models; and configuring a matched target neural network model for at least one optical waveguide combiner based on a corresponding relationship between each target neural network model and each category of the optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected through the matched target neural network model in advance. According to the method, the display quality of virtual images output by different optical waveguide combiners can be improved.

Description

Technical Field

[0001] The present application relates to the field of extended reality technology, and in particular to a model configuration method, device, system and medium based on an optical waveguide combiner. Background Art

[0002] Existing near-eye display devices such as augmented reality (AR) and mixed reality (MR) often include optical waveguide combiners. The function of the optical waveguide combiner is to transmit the image generated by the optical machine module to the human eye to present a virtual image. The user can observe the real-world scene while seeing the virtual image and can even interact with it.

[0003] However, the beam transmission performance of current waveguide combiners is limited by the manufacturing process, and beams of different field angles undergo different propagation paths after being coupled to the waveguide combiner. These factors are difficult to correct when designing waveguide combiners, so the display effects of virtual images presented by transmission through different waveguide combiners vary greatly, which may reduce the display quality of virtual images. Summary of the invention

[0004] The main purpose of the present application is to provide a model configuration method, device, system and medium based on an optical waveguide combiner, and specifically to provide a model configuration method, computer device, model configuration system and storage medium based on an optical waveguide combiner. The present application aims to solve the problem that the display effects of virtual images presented by transmission through different optical waveguide combiners are quite different, which may reduce the display quality of the virtual images.

[0005] In a first aspect, the present application provides a model configuration method, comprising:

[0006] Acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners;

[0007] Inputting each of the sample image sets into the optical waveguide combiner of the corresponding category for processing to obtain a plurality of output image sets;

[0008] Iteratively training multiple neural network models according to the multiple sample image sets and the multiple output image sets to obtain multiple trained target neural network models; wherein the multiple neural network models correspond one-to-one to the multiple categories of the optical waveguide combiner;

[0009] Based on the correspondence between each of the target neural network models and each category of the optical waveguide combiner, a matching target neural network model is configured for a plurality of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is corrected in advance by the matching target neural network model.

[0010] In a second aspect, the present application also provides a model configuration method based on an optical waveguide combiner, comprising:

[0011] Acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners;

[0012] Inputting each of the sample image sets into the optical waveguide combiner of the corresponding category for processing to obtain a plurality of output image sets;

[0013] Iteratively training a preset neural network model according to the plurality of sample image sets and the plurality of output image sets to obtain a trained target neural network model;

[0014] The target neural network model is configured for multiple optical waveguide combiners so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the target neural network model.

[0015] In a third aspect, the present application also provides a model configuration method based on an optical waveguide combiner, comprising:

[0016] Acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners;

[0017] Inputting each of the sample image sets into the optical waveguide combiner of the corresponding category for processing to obtain a plurality of output image sets;

[0018] Iteratively training a preset neural network model according to the plurality of sample image sets, the plurality of output image sets and the optical performance parameters of the optical waveguide combiners of multiple categories to obtain a trained target neural network model;

[0019] The target neural network model is configured for multiple optical waveguide combiners so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by a matching target neural network model.

[0020] In a fourth aspect, the present application also provides a computer device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing a connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, a model configuration method as described in any one of the embodiments of the present application is implemented.

[0021] In a fifth aspect, the present application further provides a model configuration system, the model configuration system comprising:

[0022] a plurality of near-eye display devices, the near-eye display devices comprising an optical waveguide combiner and a storage device;

[0023] As described in the embodiment of the present application, the computer device is communicatively connected to the plurality of near-eye display devices, and the computer device is used to determine a target neural network model that matches the optical waveguide combiner in each of the near-eye display devices, and send the matched target neural network model to a storage device in each of the near-eye display devices, so that when each of the optical waveguide combiners transmits a virtual image, the target neural network model in the corresponding storage device is pre-called to correct the virtual image.

[0024] In a sixth aspect, the present application also provides a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the model configuration method described in any one of the embodiments of the present application.

[0025] The embodiment of the present application provides a model configuration method based on an optical waveguide combiner, and a matching target neural network model is trained for optical waveguide combiners of different categories, and multiple categories of optical waveguide combiners are determined according to the optical performance parameters of multiple optical waveguide combiners. Therefore, it is possible to configure a target neural network model for optical waveguide combiners with different optical performance parameters, so that the virtual image can be corrected in advance by the matching target neural network model, and then the corrected virtual image is transmitted by the optical waveguide combiner, so that the display quality of the virtual image output by different optical waveguide combiners can be improved. Therefore, it is possible to solve the problem that the display effects of virtual images presented by transmission through different optical waveguide combiners are greatly different, which may reduce the display quality of the virtual image. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 This is an application scenario diagram of the optical waveguide combiner provided in the embodiment of the present application;

[0028] Figure 2 A schematic flow chart of the steps of a model configuration method based on an optical waveguide combiner provided in an embodiment of the present application;

[0029] Figure 3 This is an application scenario diagram of a near-eye display device provided in an embodiment of the present application;

[0030] Figure 4 It is an application scenario diagram for implementing the model configuration method provided in the embodiment of the present application;

[0031] Figure 5 It is another application scenario diagram for implementing the model configuration method provided in the embodiment of the present application;

[0032] Figure 6 This is a schematic diagram of a scenario for implementing the model configuration method provided in the embodiment of the present application;

[0033] Figure 7 A schematic flow chart of another method for configuring a model based on an optical waveguide combiner provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of another scenario for implementing the model configuration method provided in an embodiment of the present application;

[0035] Fig. 9 A schematic flow chart of steps of another model configuration method based on an optical waveguide combiner provided in an embodiment of the present application;

[0036] Fig.10 A schematic diagram of another scenario for implementing the model configuration method provided in the embodiment of the present application;

[0037] Fig.11 A schematic block diagram of a computer device provided in an embodiment of the present application;

[0038] Fig.12 A schematic block diagram of a model configuration system provided in an embodiment of the present application.

[0039] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application; it is obvious that the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0042] The optical waveguide combiner involved in the embodiments of the present application is a device that can couple the mode of the transmitted light wave with the light energy. Optical waveguide combiners are divided into various types, including directional couplers, grating couplers, Y-splitters, multimode interference couplers, etc. These couplers work between different optical waveguides. The optical waveguide combiner may include components such as coupling-in modules and coupling-out modules.

[0043] The near-eye display device involved in the embodiment of the present application is a glass or goggle-type wearable display device, which is composed of a micro display panel and an imaging optical device. The near-eye display device is close to the eye, and the light emitted by the micro display panel is collimated by the imaging optical device so that a virtual image is formed at a long distance where the eye can focus comfortably. The imaging optical device may include an optical waveguide combiner.

[0044] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0045] Figure 1 FIG. 1 is an application scenario diagram of the optical waveguide combiner provided in the embodiment of the present application. Figure 1 As shown, the light source 10 emits a light beam carrying image information, which is then coupled into the optical waveguide combiner 30 through the coupling module 20. The light coupled into the optical waveguide combiner 30 is transmitted in the optical waveguide combiner 30 at an angle that satisfies the total internal reflection condition until it is transmitted to the coupling module 40 and emitted to the human eye.

[0046] However, the optical waveguide combiner 30 is limited by the actual manufacturing process, and its surface flatness will affect the phase, polarization and other parameters of the light transmitted in the optical waveguide combiner 30. Therefore, the light beams of different viewing angles coupled to the optical waveguide combiner 30 experience different propagation paths, which will lead to differences in the optical performance of each optical waveguide combiner 30. These factors are difficult to correct when designing the optical waveguide combiner 30, so the display effect of the virtual image presented to the human eye will be reduced.

[0047] Based on this, the embodiment of the present application provides a model configuration method based on an optical waveguide combiner, a computer device, a model configuration system and a storage medium. Among them, the model configuration method based on an optical waveguide combiner can be applied to a computer device, and the computer device may include a terminal device or a server. The terminal device may be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant and a wearable device. The wearable device may include a near-eye display device. The server may be a single server or a server cluster composed of multiple servers.

[0048] Please refer to Figure 2 , Figure 2 A schematic flow chart of the steps of a model configuration method based on an optical waveguide combiner provided in an embodiment of the present application, the model configuration method comprising:

[0049] S101, acquiring a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners.

[0050] The sample image set includes multiple sample images, and the multiple sample images are used as a training sample set of the neural network model. For different types of optical waveguide combiners, corresponding sample image sets can be established, and each type of optical waveguide combiner corresponds to multiple sample image sets, and the sample image sets corresponding to different types of optical waveguide combiners can be the same or different.

[0051] Since different optical waveguide combiners have different optical performance parameters, the optical transmission performance between different optical waveguide combiners may be greatly different, so that the display effects of virtual images presented by transmission through different optical waveguide combiners are greatly different.

[0052] Therefore, the optical waveguide combiner can be divided into multiple categories, and the multiple categories of the optical waveguide combiner can be determined according to the optical performance parameters of the multiple optical waveguide combiners. The optical transmission performance of the optical waveguide combiner can be determined by the optical performance parameters of the optical waveguide combiner. The optical transmission performance of the optical waveguide combiner of the same category is similar, and the optical transmission performance of the optical waveguide combiner of different categories is different.

[0053] In one embodiment, the optical performance parameters may include parameters such as modulation transfer function (MTF), ghosting, diffraction efficiency, uniformity, etc. The optical performance parameters may also include parameters such as aperture function Q of input grating, transmission function H of waveguide combiner, aperture function P of output grating, etc. Therefore, based on the optical performance parameters of optical waveguide combiners, optical waveguide combiners with large differences in light transmission performance may be divided into multiple categories.

[0054] Exemplarily, optical performance parameters of a plurality of optical waveguide combiners are obtained; based on the optical performance parameters of the plurality of optical waveguide combiners, the plurality of optical waveguide combiners are evaluated to obtain evaluation results of the plurality of optical waveguide combiners; based on the evaluation results of the plurality of optical waveguide combiners, the plurality of optical waveguide combiners are classified to obtain a plurality of categories of the optical waveguide combiners.

[0055] The evaluation of the multiple optical waveguide combiners may be a rating or a score, and the evaluation result may be a rating result or a score result, and the specific evaluation method may be flexibly set according to the actual situation. For example, the optical performance parameters include MTF, ghosting, diffraction efficiency, and uniformity, and the score values ​​of the multiple optical waveguide combiners are determined based on the values ​​of the MTF, ghosting, diffraction efficiency, and uniformity; and the multiple optical waveguide combiners are divided into multiple different categories such as A, B, C, and D according to the score value of each optical waveguide combiner. The score values ​​of the optical waveguide combiners of the same category are in the same set score segment, and the score values ​​of the optical waveguide combiners of different categories are in different set score segments.

[0056] Exemplarily, optical performance parameters of a plurality of optical waveguide combiners are obtained; based on the optical performance parameters of the plurality of optical waveguide combiners, the plurality of optical waveguide combiners are evaluated to obtain evaluation results of the plurality of optical waveguide combiners; based on the evaluation results of the plurality of optical waveguide combiners, the plurality of optical waveguide combiners are classified to obtain a plurality of categories of the optical waveguide combiners.

[0057] The optical performance parameters include the aperture function Q of the input grating and the waveguide combiner transmission function H. Based on the optical performance parameters, the score values ​​or ratings of the plurality of optical waveguide combiners can be determined. According to the score value or rating of each optical waveguide combiner, the plurality of optical waveguide combiners can be divided into a plurality of different categories such as A, B, C, and D.

[0058] S102, inputting each sample image set into a corresponding type of optical waveguide combiner for processing to obtain a plurality of output image sets.

[0059] It should be noted that each sample image set needs to be processed based on the actually manufactured optical waveguide combiner to obtain the corresponding output image set.

[0060] In one embodiment, a near-eye display device includes an image source module, an optical waveguide combiner, and a detection module. A plurality of sample images in a sample image set are sequentially loaded into an optical signal of the image source module, the image source module outputs an optical signal loaded with the sample image to the optical waveguide combiner, and transmits the optical signal to the detection module through the optical waveguide combiner, the output signal of the optical waveguide combiner is collected through the detection module, and an output image in the output signal is extracted to obtain an output image set.

[0061] For example, Figure 3As shown, the near-eye display device includes an image source module 31, an optical waveguide combiner 30 and a detection module 32. The input signal (sample image) is sequentially loaded onto the optical signal on the image source module 31, and then the optical signal carrying the sample image information is transmitted to the detection module 40 through the optical waveguide combiner 30. The output signal of the optical waveguide combiner 30 is collected by the detection module 40 and the output image therein is extracted to obtain an output image set.

[0062] S103, iteratively training multiple neural network models according to multiple sample image sets and multiple output image sets to obtain multiple trained target neural network models.

[0063] Among them, multiple neural network models correspond to multiple categories of optical waveguide combiners one by one. It should be noted that the output image set is obtained by inputting the sample image set into the optical waveguide combiner of the corresponding category for processing. Therefore, by using the neural network model to simulate the optical waveguide combiner and iteratively training the neural network model based on the sample image set and the output image set, the trained target neural network model can have the optical transmission characteristics of the corresponding optical waveguide combiner.

[0064] It should be noted that after obtaining multiple sample image sets and multiple output image sets, the neural network model can be pre-established based on multiple categories of optical waveguide combiners, so multiple sample image sets and multiple output image sets can be directly used to iteratively train multiple neural network models. The neural network model can include a convolutional neural network CNN, a recurrent neural network RNN, a deep neural network DNN, etc., and can also be other types of neural network models such as a forward differentiable model, which is not specifically limited in this application.

[0065] In one embodiment, before iteratively training multiple neural network models, it is necessary to first establish multiple neural network models corresponding to multiple categories of optical waveguide combiners. Specifically, before iteratively training multiple neural network models according to multiple sample image sets and multiple output image sets to obtain multiple trained target neural network models, it also includes: determining the first configuration parameters of each neural network model according to the optical performance parameters of each category of optical waveguide combiners; determining the second configuration parameters of each neural network model according to the image size of each sample image set or each output image set; and constructing multiple neural network models based on each first configuration parameter and each second configuration parameter.

[0066] Among them, the optical performance parameters of the optical waveguide combiner include the aperture function of the input grating, the waveguide combiner transmission function and the aperture function of the output grating, and of course other parameters may also be included. The optical performance parameters can be used as hyperparameters of the neural network model without participating in training, so the first configuration parameter can include the hyperparameters of the neural network model. The larger the image size of the sample image set or the output image set, the more levels of the neural network model, such as the more hidden layers, so the second configuration parameter can be a scale parameter for characterizing the neural network model. Based on the first configuration parameter and the second configuration parameter, a neural network model corresponding to multiple categories of optical waveguide combiners can be specifically constructed, thereby improving the matching degree between the neural network model and the corresponding category of optical waveguide combiners, and improving the reliability of the neural network model in practical applications.

[0067] In one embodiment, the training process of the neural network model includes: inputting a first sample image in a sample image set into a corresponding neural network model for processing to obtain a first output image; calculating a first error value between the first output image and a corresponding output image in the output image set; when the first error value is greater than a preset error value, updating the model parameters of the neural network model according to the first error value, and iteratively training the neural network model with updated model parameters; when the first error value is less than or equal to the preset error value, determining the neural network model with updated model parameters as the trained target neural network model.

[0068] The first sample image can be any sample image in the sample image set, and the first sample image is used as the input of the neural network model, and the first output image is obtained after being processed by the neural network model. Since the model parameters of the neural network model are initially random, there is a difference between the predicted result of the neural network model (the first output image) and the actual result (the corresponding output image in the output image set) obtained by processing the corresponding type of optical waveguide combiner.

[0069] It should be noted that the first error value is the difference between the predicted result and the actual result of the neural network model, such as the peak signal-to-noise ratio, etc. According to the first error value, the model parameters of the neural network model can be updated based on the back propagation algorithm. Iterative training of the neural network model with updated model parameters can be the step of continuing to input the first sample image in the sample image set into the corresponding neural network model for processing to obtain the first output image. After training N groups of images, the difference between the predicted result and the actual result of the neural network model is less than the preset error value. At this time, the model parameter training is completed, and the trained target neural network model is obtained.

[0070] For example, Figure 4As shown, a first sample image 41 is input into the neural network model, and a first output image 42 is obtained at the output end of the neural network model. A first error value 44 between the first output image 42 and the corresponding output image 43 in the output image set is calculated, and the model parameters are updated according to the first error value 44 and the back propagation algorithm. After iterative training of the neural network model with N groups of first sample images 41, the first error value 44 is less than the preset error value, and the model parameter training is completed. Finally, the prediction accuracy of the neural network model can also be tested with test image data.

[0071] In another embodiment, it is also possible to determine whether a trained target neural network model is obtained based on parameters such as the number of iterative training times and the iterative training duration of the neural network model, which is not specifically limited in this application. For example, when the number of iterative training times is greater than a preset number of training times, or when the iterative training duration is greater than a preset duration, the currently trained neural network model is determined as the trained target neural network model.

[0072] In one embodiment, the model configuration method also includes: inputting a second sample image in the sample image set into a trained target neural network model for processing to obtain a second output image; calculating a second error value between the second output image and the corresponding expected output image; and based on the second error value, updating the second sample image using a back propagation algorithm so that the second error value between the updated second sample image and the expected output image is less than or equal to a preset error value.

[0073] The second sample image may be any sample image in the sample image set, and the expected output image may be the best display image of the second sample image, which may be set and have the best display effect. The second error value may be a parameter such as a peak signal-to-noise ratio. Based on the second error value and the back propagation algorithm, the second sample image is updated, and the optimization of the second sample image can be achieved, so that the updated second sample image is closer to the real output image of the optical waveguide combiner after being transmitted by the optical waveguide combiner.

[0074] It should be noted that, based on the trained target neural network model, the sample images in the sample image set can be optimized, and the virtual images presented by the optimized sample images transmitted through different optical waveguide combiners have the best display effect.

[0075] For example, Figure 5As shown, based on the trained target neural network model, a second sample image 51 is randomly input, and the output result obtained by the target neural network model is a second output image 52, which is consistent with the output image output by the optical waveguide combiner. This is because after training with multiple sample image sets, the target neural network model already has all the characteristics of the optical waveguide combiner. At this time, consistent with the problem of output images output by the optical waveguide combiner, the second output image 52 will also have the problem of reduced display effect. Then, for the output result, that is, the second output image 52, the second error value 54 between it and the expected output image 53 is calculated. And the second sample image 51 is updated according to the back propagation algorithm until the second error value 54 between the second output image 52 and the expected output image 53 is less than or equal to the preset error value. At this time, the updated sample image 55 obtained is an optimized image that can be used for the optical waveguide combiner.

[0076] S104, based on the correspondence between each target neural network model and each category of the optical waveguide combiner, a matching target neural network model is configured for multiple optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the matching target neural network model.

[0077] It should be noted that, based on the aforementioned steps, the correspondence between each target neural network model and each category of the optical waveguide combiner can be obtained, and based on the correspondence, a target neural network model matching the target neural network model can be configured for multiple optical waveguide combiners. Among them, configuring a matching target neural network model for multiple optical waveguide combiners can refer to storing each target neural network model in a storage device corresponding to each optical waveguide combiner, so that when each optical waveguide combiner transmits a virtual image, the target neural network model in the corresponding storage device is pre-called to correct the virtual image. The storage device can be integrated with the optical waveguide combiner in a near-eye display device, and the storage device can also be a cloud server.

[0078] It should be noted that when a certain type of optical waveguide combiner is actually used, the input image must first be corrected and optimized according to the configured target neural network model, and then transmitted to the human eye through the optical waveguide combiner, so as to ensure that different types of optical waveguide combiners can participate in the correction of virtual images through the matching target neural network model in advance, so that different types of optical waveguide combiners can have their own optimal display effects. For example, in the process of outputting a virtual image, the virtual image is first corrected by the matching target neural network model, and then the corrected virtual image is transmitted by the optical waveguide combiner.

[0079] Exemplarily, the near-eye display device includes an optical waveguide combiner and a storage device. Based on the correspondence between each target neural network model and each category of the optical waveguide combiner, a matching target neural network model is configured for multiple optical waveguide combiners; the target neural network model matching the optical waveguide combiner in each near-eye display device is determined, and the matching target neural network model is sent to the storage device in each near-eye display device, so that when each optical waveguide combiner transmits a virtual image, the target neural network model in the corresponding storage device is pre-called to correct the virtual image.

[0080] In one embodiment, a mapping relationship table is obtained, wherein the mapping relationship table records the correspondence between the categories of multiple optical waveguide combiners and multiple trained target neural network models; the target neural network models that match each of the multiple optical waveguide combiners are determined based on the mapping relationship table; and each target neural network model is stored in a storage device corresponding to each optical waveguide combiner.

[0081] Among them, the correspondence between the categories of multiple optical waveguide combiners and multiple trained target neural network models is recorded to obtain a mapping relationship table. The mapping relationship table can be stored in the memory of a computer device. When a certain type of optical waveguide combiner needs to be applied, the target neural network model matched by a certain type of optical waveguide combiner can be determined based on the mapping relationship table, and each target neural network model can be stored in a storage device corresponding to the optical waveguide combiner, so that the input image can be corrected using the target neural network model first, and then the corresponding virtual image can be obtained through transmission by the optical waveguide combiner. The virtual image is used to be transmitted to the human eye, and the updating method of the input image can refer to the relevant example of the second sample image in the above embodiment, and this application does not make specific limitations on this.

[0082] Please refer to Figure 6 , Figure 6 A schematic diagram of a scenario for implementing the model configuration method provided in an embodiment of the present application.

[0083] like Figure 6 As shown, the neural network model includes models 1 to 9, and models 1 to 9 correspond to categories 1 to 9 of optical waveguide combiners, respectively. When a certain type of optical waveguide combiner needs to be applied, a matching target neural network model can be determined from models 1 to 9 based on the category of the optical waveguide combiner. When the input image of the optical waveguide combiner is input image set 1 to input image set 9, the target neural network model can be used to update the input images in input image set 1 to input image set 9 in sequence, and then the corresponding virtual image is obtained through the transmission of the optical waveguide combiner and projected to the human eye, thereby ensuring that optical waveguide combiners of different categories can have their own optimal display effects, greatly improving the display quality of the virtual image.

[0084] The model configuration method based on the optical waveguide combiner provided in the above embodiment trains matching target neural network models for optical waveguide combiners of different categories, so that target neural network models can be configured for optical waveguide combiners with different optical transmission performances to output virtual images, thereby achieving the best display effect of virtual images for different optical waveguide combiners. Therefore, the problem that the display effects of virtual images presented by transmission through different optical waveguide combiners are greatly different, which may reduce the display quality of virtual images, can be solved.

[0085] Please refer to Figure 7 , Figure 7 A schematic flow chart of the steps of another model configuration method based on an optical waveguide combiner provided in an embodiment of the present application.

[0086] like Figure 7 As shown, the model configuration method includes steps S201 to S204.

[0087] Step S201: Acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners.

[0088] The sample image set includes a plurality of sample images, and the plurality of categories of optical waveguide combiners are determined according to the optical performance parameters of the plurality of optical waveguide combiners. For optical waveguide combiners of different categories, corresponding sample image sets can be established, and each category of optical waveguide combiners corresponds to at least one sample image set, and the sample image sets corresponding to optical waveguide combiners of different categories can be the same or different.

[0089] The optical performance parameters may include modulation transfer function, ghosting, diffraction efficiency, uniformity, etc. The optical performance parameters may also include aperture function Q of input grating, waveguide combiner transmission function H, aperture function P of output grating, etc.

[0090] Step S202: Input each sample image set into a corresponding type of optical waveguide combiner for processing to obtain a plurality of output image sets.

[0091] Each sample image set needs to be processed based on an actually manufactured optical waveguide combiner to obtain its corresponding output image set. Therefore, based on the correspondence between the sample image set and the category of the optical waveguide combiner, multiple sample images in the sample image set are sequentially input into the optical waveguide combiner of the corresponding category for processing to obtain the output image set of the sample image set.

[0092] Step S203: Iteratively train the preset neural network model according to the multiple sample image sets and the multiple output image sets to obtain a trained target neural network model.

[0093] The pre-set neural network model can be a large model. The large model can be used to simulate multiple categories of optical waveguide combiners. Based on multiple sample image sets and multiple output image sets, the pre-set neural network model is iteratively trained, and the trained target neural network model can have the optical transmission characteristics of multiple categories of optical waveguide combiners.

[0094] In one embodiment, the method for constructing the pre-set neural network model includes: determining the first configuration parameter of the neural network model according to the optical performance parameters of each category of optical waveguide combiner; determining the second configuration parameter of the neural network model according to the image size of each sample image set or each output image set; and constructing the neural network model based on each first configuration parameter and each second configuration parameter.

[0095] The optical performance parameters of the optical waveguide combiner include the aperture function of the input grating, the waveguide combiner transmission function and the aperture function of the output grating, and of course other parameters may also be included. The optical performance parameters can be used as hyperparameters of the neural network model without participating in training, so the first configuration parameter can be the average value or weighted average value of the optical performance parameters. The larger the image size of the sample image set or the output image set, the more levels of the neural network model, such as the more hidden layers, so the second configuration parameter can be a scale parameter used to characterize the neural network model, and the second configuration parameter can be determined according to the maximum image size of the sample image set or the output image set.

[0096] It should be noted that, based on the first configuration parameter and the second configuration parameter, a neural network model matching multiple categories of optical waveguide combiners can be specifically constructed, thereby improving the matching degree between the neural network model and different categories of optical waveguide combiners and improving the reliability of the neural network model in practical applications.

[0097] It can be understood that the training process of the neural network model can refer to the specific training process of the neural network model in the above embodiment, and the embodiments of the present application will not go into details.

[0098] Step S204: configuring a target neural network model for the plurality of optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the target neural network model.

[0099] Among them, a large-model target neural network model is trained for different types of optical waveguide combiners, and the respective input images can be pre-corrected for different optical waveguide combiners based on the large-model target neural network model, so that different types of optical waveguide combiners can have good display effects, thereby improving the display quality of virtual images. Therefore, it can solve the problem that the display effects of virtual images presented by transmission through different optical waveguide combiners are greatly different, which may reduce the display quality of virtual images.

[0100] In one embodiment, the target neural network model is stored in storage devices corresponding to a plurality of optical waveguide combiners. When a certain type of optical waveguide combiner is required, the input image can be pre-corrected using the target neural network model in the storage device, and then the corresponding virtual image is obtained through transmission of the optical waveguide combiner, so that different types of optical waveguide combiners can have better display effects.

[0101] Exemplarily, the near-eye display device includes an optical waveguide combiner and a storage device. The target neural network model is configured for different types of optical waveguide combiners; the target neural network model is sent to the storage device in each near-eye display device, so that when each optical waveguide combiner transmits a virtual image, the target neural network model in the corresponding storage device is pre-called to correct the virtual image.

[0102] Please refer to Figure 8 , Figure 8 A schematic diagram of another scenario for implementing the model configuration method provided in an embodiment of the present application.

[0103] like Figure 8 As shown, when a certain type of optical waveguide combiner needs to be applied, if the input image of the optical waveguide combiner is input image set 1 to input image set 9, the input images in input image set 1 to input image set 9 can be corrected in sequence based on the large model (target neural network model), and then the corresponding virtual image is obtained through transmission by the optical waveguide combiner and projected to the human eye, thereby ensuring that different types of optical waveguide combiners can have a good display effect, thereby improving the display quality of the virtual image.

[0104] Please refer to Fig. 9 , Fig. 9 A schematic flow chart of the steps of another model configuration method based on an optical waveguide combiner provided in an embodiment of the present application.

[0105] like Fig. 9 As shown, the model configuration method includes steps S301 to S304.

[0106] Step S301: Acquire multiple sample image sets, wherein the multiple sample image sets correspond one-to-one to multiple categories of optical waveguide combiners.

[0107] The sample image set includes a plurality of sample images, and the plurality of categories of optical waveguide combiners are determined according to the optical performance parameters of the plurality of optical waveguide combiners. For optical waveguide combiners of different categories, corresponding sample image sets can be established, and each category of optical waveguide combiners corresponds to at least one sample image set, and the sample image sets corresponding to optical waveguide combiners of different categories can be the same or different.

[0108] Step S302: Input each sample image set into a corresponding type of optical waveguide combiner for processing to obtain a plurality of output image sets.

[0109] Each sample image set needs to be processed based on an actually manufactured optical waveguide combiner to obtain its corresponding output image set. Therefore, based on the correspondence between the sample image set and the category of the optical waveguide combiner, multiple sample images in the sample image set are sequentially input into the optical waveguide combiner of the corresponding category for processing to obtain the output image set of the sample image set.

[0110] Step S303: iteratively train the preset neural network model according to the multiple sample image sets, the multiple output image sets and the optical performance parameters of the multiple categories of optical waveguide combiners to obtain a trained target neural network model.

[0111] The pre-set neural network model can be a large model. The large model can be used to simulate multiple categories of optical waveguide combiners. When training the large model (pre-set neural network model), the optical performance parameters of multiple categories of optical waveguide combiners are input into the network as hyperparameters of the model for training together. When the trained large model (target neural network) is used, the large model can be calibrated by the optical performance parameters of the optical waveguide combiner actually measured, so that the output result is more accurate.

[0112] It should be noted that based on multiple sample image sets, multiple output image sets and multiple optical performance parameters, the pre-set neural network model is iteratively trained. The trained target neural network model can possess the optical transmission characteristics of multiple categories of optical waveguide combiners, and the output results are more accurate.

[0113] In one embodiment, the method for constructing the pre-set neural network model includes: determining the first configuration parameter of the neural network model according to the optical performance parameters of each category of optical waveguide combiner; determining the second configuration parameter of the neural network model according to the image size of each sample image set or each output image set; and constructing the neural network model based on each first configuration parameter and each second configuration parameter.

[0114] Among them, the optical performance parameters of the optical waveguide combiner include MTF, ghosting, diffraction efficiency, uniformity, etc., and of course other parameters may also be included. The optical performance parameter can be used as a hyperparameter of the neural network model without participating in training, so the first configuration parameter can be the average value or weighted average value of the optical performance parameter. The larger the image size of the sample image set or the output image set, the more levels of the neural network model, such as the more hidden layers, so the second configuration parameter can be a scale parameter used to characterize the neural network model, and the second configuration parameter can be determined according to the maximum image size of the sample image set or the output image set.

[0115] It should be noted that, based on the first configuration parameter and the second configuration parameter, a neural network model matching multiple categories of optical waveguide combiners can be specifically constructed, thereby improving the matching degree between the neural network model and different categories of optical waveguide combiners and improving the reliability of the neural network model in practical applications.

[0116] In one embodiment, a third sample image in the sample image set and optical performance parameters of a corresponding category of optical waveguide combiner are input into a preset neural network model for processing to obtain a third output image; a third error value between the third output image and the corresponding output image in the output image set is calculated; and model parameters of the neural network model are updated according to the third error value until the neural network model with updated model parameters converges to obtain a trained target neural network model.

[0117] The third sample image may be any sample image in the sample image set, and the third sample image and the optical performance parameters of the corresponding optical waveguide combiner are used as inputs of the neural network model, and the third output image is obtained after being processed by the neural network model. Since the model parameters of the neural network model are initially random, there is a difference between the predicted result of the neural network model (the third output image) and the actual result (the corresponding output image in the output image set) obtained after being processed by the optical waveguide combiner of the corresponding category.

[0118] It should be noted that the third error value is the difference between the predicted result and the actual result of the neural network model, such as the peak signal-to-noise ratio, etc. According to the third error value, the model parameters of the neural network model can be updated based on the back propagation algorithm. After the neural network is trained with N groups of images, the difference between the predicted result and the actual result of the neural network model is less than the preset error value. At this time, the model parameter training is completed, and the trained target neural network model is obtained.

[0119] It can be understood that the convergence condition of the network model can be that the third error value is less than or equal to the preset error value, or the number of iterative training times is greater than the preset number of training times, or the iterative training time is greater than the preset time, etc. For example, when the number of iterative training times is greater than the preset number of training times, or the iterative training time is greater than the preset time, it is determined that the neural network model converges.

[0120] Step S304: configure a target neural network model for the plurality of optical waveguide combiners, so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the matching target neural network model.

[0121] Among them, a large-model target neural network model is trained for different types of optical waveguide combiners, and the respective input images can be pre-corrected for different optical waveguide combiners based on the large-model target neural network model, so that different types of optical waveguide combiners can have a good display effect after transmitting the corrected virtual image, thereby improving the display quality of the virtual image. Therefore, it can solve the problem that the display effects of virtual images presented by transmission through different optical waveguide combiners are greatly different, which may reduce the display quality of the virtual image.

[0122] In one embodiment, the target neural network model is stored in storage devices corresponding to multiple optical waveguide combiners. When a certain type of optical waveguide combiner needs to be applied, the input image can be corrected using the target neural network model in the storage device, and then the corresponding virtual image is obtained through transmission through the optical waveguide combiner, which can ensure that different types of optical waveguide combiners can have a good display effect.

[0123] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described model configuration method can refer to the corresponding process in the aforementioned model configuration method embodiment, and will not be repeated here.

[0124] Please refer to Fig.10 , Fig.10 A schematic diagram of another scenario for implementing the model configuration method provided in an embodiment of the present application.

[0125] like Fig.10As shown, when a certain type of optical waveguide combiner needs to be applied, if the input images of the optical waveguide combiner are input image set 1 to input image set 9, and the input parameters (optical performance parameters) include MTF, ghosting, diffraction efficiency, uniformity, etc., the input images in input image set 1 to input image set 9 can be updated in sequence based on the large model (target neural network model), and then the corresponding virtual images are obtained through transmission by the optical waveguide combiner and projected to the human eye, so as to ensure that different types of optical waveguide combiners can have good display effects, thereby improving the display quality of the virtual image.

[0126] See also Fig.11 , Fig.11 A schematic block diagram of a computer device provided in an embodiment of the present application.

[0127] like Fig.11 As shown, the computer device 400 includes: a processor 410 and a memory 420, and the processor 410 and the memory 420 are connected via a bus 430, such as an I2C (Inter-integrated Circuit) bus.

[0128] Specifically, the processor 410 is used to provide computing and control capabilities to support the operation of the entire computer device. The processor 410 can be a central processing unit (CPU), and the processor 410 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0129] Specifically, the memory 420 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0130] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the computer device to which the embodiment of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0131] The processor is used to run a computer program stored in the memory, and implement any one of the model configuration methods provided in the embodiments of the present application when executing the computer program.

[0132] In one embodiment, the processor is used to run a computer program stored in a memory, and implement the following steps when executing the computer program: acquiring a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of the optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners; inputting each of the sample image sets into an optical waveguide combiner of a corresponding category for processing to obtain a plurality of output image sets; iteratively training a plurality of neural network models according to the plurality of sample image sets and the plurality of output image sets to obtain a plurality of trained target neural network models; wherein the plurality of neural network models correspond one-to-one to a plurality of categories of the optical waveguide combiners; and configuring a matching target neural network model for the plurality of optical waveguide combiners based on the corresponding relationship between each of the target neural network models and each category of the optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is pre-corrected by the matching target neural network model.

[0133] In one embodiment, the processor is used to run a computer program stored in a memory, and implement the following steps when executing the computer program: acquiring a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of the optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners; inputting each of the sample image sets into an optical waveguide combiner of a corresponding category for processing to obtain a plurality of output image sets; iteratively training a pre-set neural network model according to the plurality of sample image sets and the plurality of output image sets to obtain a trained target neural network model; configuring the target neural network model for a plurality of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is pre-corrected by the target neural network model.

[0134] In one embodiment, the processor is used to run a computer program stored in a memory, and implement the following steps when executing the computer program: acquiring a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of a plurality of optical waveguide combiners; inputting each of the sample image sets into an optical waveguide combiner of a corresponding category for processing to obtain a plurality of output image sets; iteratively training a pre-set neural network model according to the plurality of sample image sets, the plurality of output image sets, and the optical performance parameters of the plurality of categories of optical waveguide combiners to obtain a trained target neural network model; configuring the target neural network model for a plurality of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is pre-corrected by the target neural network model.

[0135] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the computer device 400 described above can refer to the corresponding process in the aforementioned model configuration method embodiment, and will not be repeated here.

[0136] See also Fig.12 , Fig.12 A schematic block diagram of the structure of a model configuration system provided in an embodiment of the present application.

[0137] like Fig.12 As shown, the model configuration system 500 includes: a plurality of near-eye display devices 510 and a computer device 520 .

[0138] The near-eye display device 510 includes an optical waveguide combiner and a storage device. The computer device 520 is connected to each near-eye display device 510. The computer device 520 is used to determine the target neural network model that matches the optical waveguide combiner in each near-eye display device 510, and send the matched target neural network model to the storage device in each near-eye display device 510, so that when each optical waveguide combiner transmits a virtual image, the target neural network model in the corresponding storage device is called in advance to correct the virtual image.

[0139] In some embodiments, the near-eye display device 510 may include an augmented reality (AR) display device, a mixed reality (MR) display device, etc. The computer device 520 may be the computer device 400 in the aforementioned embodiment.

[0140] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the model configuration system 500 described above can refer to the corresponding process in the aforementioned model configuration method embodiment, and will not be repeated here.

[0141] The embodiments of the present application also provide a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any model configuration method based on an optical waveguide combiner as provided in the embodiments of the present application.

[0142] The storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A model configuration method based on an optical waveguide combiner, characterized in that: include: Acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners; Inputting each of the sample image sets into the optical waveguide combiner of the corresponding category for processing to obtain a plurality of output image sets; Iteratively training multiple neural network models according to the multiple sample image sets and the multiple output image sets to obtain multiple trained target neural network models; wherein the multiple neural network models correspond one-to-one to the multiple categories of the optical waveguide combiner; Based on the correspondence between each of the target neural network models and each category of the optical waveguide combiner, a matching target neural network model is configured for a plurality of optical waveguide combiners, so that when each of the optical waveguide combiners transmits a virtual image, the virtual image is corrected in advance by the matching target neural network model.

2. The model configuration method according to claim 1, characterized in that: The iterative training of multiple neural network models according to the multiple sample image sets and the multiple output image sets to obtain multiple trained target neural network models includes: Inputting a first sample image in the sample image set into a corresponding neural network model for processing to obtain a first output image; Calculating a first error value between the first output image and a corresponding output image in the output image set; When the first error value is greater than a preset error value, updating a model parameter of the neural network model according to the first error value, and iteratively training the neural network model with the updated model parameters; When the first error value is less than or equal to the preset error value, the neural network model whose model parameters are updated is determined as the trained target neural network model.

3. The model configuration method according to claim 2, characterized in that: The method further comprises: Inputting a second sample image in the sample image set into the trained target neural network model for processing to obtain a second output image; Calculating a second error value between the second output image and the corresponding expected output image; Based on the second error value, the second sample image is updated using a back propagation algorithm so that a second error value between the updated second sample image and the expected output image is less than or equal to the preset error value.

4. The model configuration method according to claim 1, characterized in that: Before iteratively training multiple neural network models according to the multiple sample image sets and the multiple output image sets to obtain multiple trained target neural network models, the method further includes: Determining first configuration parameters of each of the neural network models according to optical performance parameters of each of the categories of optical waveguide combiners; Determining a second configuration parameter of each of the neural network models according to an image size of each of the sample image sets or each of the output image sets; Based on each of the first configuration parameters and each of the second configuration parameters, a plurality of the neural network models are constructed.

5. The model configuration method according to claim 4, characterized in that: The optical performance parameters of the optical waveguide combiner include an aperture function of an input grating, a waveguide combiner transmission function and an aperture function of an output grating.

6. The model configuration method according to any one of claims 1 to 5, characterized in that: The configuring a matching target neural network model for a plurality of optical waveguide combiners based on the correspondence between each of the target neural network models and the category information of each of the optical waveguide combiners comprises: Generate a mapping relationship table, wherein the mapping relationship table records the corresponding relationship between the categories of multiple optical waveguide combiners and multiple trained target neural network models; Determine target neural network models that are matched to each of the plurality of optical waveguide combiners based on the mapping relationship table; Each of the target neural network models is stored in a storage device corresponding to each of the optical waveguide combiners.

7. The model configuration method according to any one of claims 1 to 5, characterized in that: The method further comprises: evaluating the plurality of optical waveguide combiners according to the optical performance parameters of the plurality of optical waveguide combiners to obtain evaluation results of the plurality of optical waveguide combiners; Based on the evaluation results of the plurality of optical waveguide combiners, the plurality of optical waveguide combiners are classified to obtain a plurality of categories of the optical waveguide combiners.

8. A model configuration method based on an optical waveguide combiner, characterized in that: include: Acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners; Inputting each of the sample image sets into the optical waveguide combiner of the corresponding category for processing to obtain a plurality of output image sets; Iteratively training a preset neural network model according to the plurality of sample image sets and the plurality of output image sets to obtain a trained target neural network model; The target neural network model is configured for multiple optical waveguide combiners so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by the target neural network model.

9. A model configuration method based on an optical waveguide combiner, characterized in that: include: Acquire a plurality of sample image sets, wherein the plurality of sample image sets correspond one-to-one to a plurality of categories of optical waveguide combiners, and the plurality of categories of optical waveguide combiners are determined according to optical performance parameters of the plurality of optical waveguide combiners; Inputting each of the sample image sets into the optical waveguide combiner of the corresponding category for processing to obtain a plurality of output image sets; Iteratively training a preset neural network model according to the plurality of sample image sets, the plurality of output image sets and the optical performance parameters of the optical waveguide combiners of multiple categories to obtain a trained target neural network model; The target neural network model is configured for multiple optical waveguide combiners so that when each optical waveguide combiner transmits a virtual image, the virtual image is corrected in advance by a matching target neural network model.

10. The model configuration method according to claim 9, characterized in that: The method of iteratively training a preset neural network model according to the plurality of sample image sets, the plurality of output image sets and the optical performance parameters of the optical waveguide combiners of multiple categories to obtain a trained target neural network model comprises: Inputting a third sample image in the sample image set and an optical performance parameter of an optical waveguide combiner of a corresponding category into a preset neural network model for processing to obtain a third output image; Calculating a third error value between the third output image and a corresponding output image in the output image set; The model parameters of the neural network model are updated according to the third error value until the neural network model with updated model parameters converges to obtain a trained target neural network model.

11. A computer device, characterized in that: The computer device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the model configuration method according to any one of claims 1 to 10 is realized.

12. A model configuration system, characterized in that: The model configuration system comprises: a plurality of near-eye display devices, the near-eye display devices comprising an optical waveguide combiner and a storage device; The computer device as described in claim 11 is communicatively connected to each of the near-eye display devices, and is used to determine a target neural network model that matches the optical waveguide combiner in each of the near-eye display devices, and send the matched target neural network model to a storage device in each of the near-eye display devices, so that when each of the optical waveguide combiners transmits a virtual image, the target neural network model in the corresponding storage device is pre-called to correct the virtual image.

13. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the model configuration method according to any one of claims 1 to 10.

Images