Integral imaging parameter optimization method based on deep learning and multi-objective genetic algorithm

Abstract

An integral imaging parameter optimization method based on deep learning and a multi-objective genetic algorithm. The method comprises: prediction by a neural network and precise optimization using a genetic algorithm; prediction by a neural network comprises establishing a relational model between three system parameters, i.e. the lens focal length and the lens pitch of a microlens array and the distance from the lens array to a display screen, and three display parameters, i.e. the resolution, the depth of field and the field of view, and using same to perform reverse prediction of rough system parameters (output) on the basis of display parameters (input), a training data set used being a data set that is obtained under random actual conditions by means of optimization based on a genetic algorithm; and precise optimization using a genetic algorithm comprises performing population processing on the combination output by the previous step, using same as an input of an SPEA2-based genetic algorithm, and performing a smaller number of iterations to obtain a precise solution.

Description

An Integrated Imaging Parameter Optimization Method Based on Deep Learning and Multi-Objective Genetic Algorithm Technical Field

[0001] This application relates to integrated imaging technology, and more specifically to an integrated imaging parameter optimization method that integrates deep learning and multi-objective genetic algorithms. Background Technology

[0002] Integrated Imaging 3D Display (IMD) technology, a type of light field 3D display technology, utilizes microlenses or pinhole arrays to image a pre-recorded or computer-rendered image array with specific parameter settings, thereby reproducing a reconstructed 3D image of the original spatial scene. Its technical principle is relatively simple, and it possesses numerous advantages such as full parallax, viewpoint continuity, full color, and 2D image preservation. It has seen significant development in recent years and is a major focus in the field of 3D display.

[0003] Due to the limitations of integrated imaging principles, display characteristics, including parameters such as resolution, depth of field, and field of view, are interdependent. Different application scenarios often require different display systems to highlight the performance of one or more parameters. Therefore, an optimization method is needed to optimize the parameters of display devices according to display parameter requirements.

[0004] In view of the above, this application aims to provide a method for establishing a network model of the correspondence between system parameters and display performance of an integrated imaging basic display system based on deep learning, and combining it with a multi-objective genetic algorithm for joint optimization, in order to solve one or more of the above-mentioned technical problems. Summary of the Invention

[0005] To address one or more technical problems in the prior art, according to one aspect of this application, an integrated imaging parameter optimization method that combines deep learning and multi-objective genetic algorithms is provided, characterized by comprising the following steps:

[0006] The following formula relates the parameters of a basic integrated imaging system display device to its main display performance parameters: θ = arctan(p / g)

[0007] The evaluation function and fitness function are defined based on the weights, introducing two additional parameters: the number of viewpoints and crosstalk, as shown in the following formula: D NCT = g(M-1)

[0008] The training dataset is randomly generated using a genetic algorithm based on a certain range of displayed parameters;

[0009] The randomly generated data sets are processed to reduce the uncertainty in the results caused by the genetic algorithm being a random algorithm;

[0010] The training dataset is created by taking the various display performance parameters in the dataset and known device parameters, such as screen resolution and pixel size, as inputs and the display device parameters as outputs.

[0011] Construct a Full Connect Neural Network (FCNN) algorithm model;

[0012] Train the FCNN network model and update the network weight parameters;

[0013] Save the network model corresponding to the FCNN with the best training result;

[0014] The target display parameters are fed into the constructed FCNN network model, and a rough system parameter value is obtained by back-prediction.

[0015] The predicted values ​​are expanded into the initial population for the genetic algorithm;

[0016] The exact solution was obtained by using the multi-objective SPEA2 genetic algorithm for optimization.

[0017] According to another aspect of this application, the FCNN neural network includes a hierarchical structure of an input layer, multiple hidden layers, and an output layer.

[0018] According to another aspect of this application, the hidden layers of the FCNN neural network are multiple fully connected layers and ReLU activation layers, and the output layer is an identity layer.

[0019] According to another aspect of this application, the training process of the FCNN network model includes:

[0020] The preprocessed training dataset is input into the FCNN network model for training;

[0021] Define the root mean square error loss function and calculate the loss;

[0022] Update the model weights and biases using the backpropagation algorithm;

[0023] Repeat the above steps until the loss function converges.

[0024] According to another aspect of this application, the optimization process of the multi-objective genetic algorithm includes:

[0025] An initial population is established based on neural network predictions.

[0026] Operations such as fitness allocation, environment selection, mating, and mutation are performed based on the SPEA2 algorithm;

[0027] Repeat the above steps until the iteration count condition is met.

[0028] According to another aspect of this application, the optimization method further includes joint optimization using deep learning and multi-objective genetic algorithms.

[0029] According to another aspect of this application, the training dataset is a preprocessed set of integrated imaging system parameters and display performance parameters.

[0030] Compared with the prior art, this application has one or more of the following technical effects:

[0031] First, this application utilizes a trained FCNN neural network for coarse parameter estimation, thereby enabling the rapid acquisition of parameter combinations close to the optimal solution in the initial stage, and improving the optimization speed of integrated imaging system parameter design.

[0032] Secondly, this application performs a series of preprocessing steps on the randomly generated optimization results when training the neural network. The generated dataset accelerates the convergence of the loss function during training and makes the entire dataset more suitable for actual parameter combinations, effectively improving the network training efficiency and practical value.

[0033] Third, this application uses a multi-objective genetic algorithm based on SPEA2 to optimize the exact solution and populate the initial population of the coarse solution, which can significantly reduce the number of iterations, shorten the optimization speed of the integrated imaging system design parameters, and achieve a fast and efficient optimization process.

[0034] Fourth, this application defines a weighted fitness function in the multi-objective genetic algorithm process, which can change the optimization direction for different application requirements and improve the practical value of parameter optimization of integrated imaging systems. Attached Figure Description

[0035] To understand the details of the above-described features of this application, a more detailed description of the invention, briefly summarized above, can be obtained by referring to the embodiments. The accompanying drawings relate to preferred embodiments of this application and are described below:

[0036] Figure 1 is a schematic diagram of the entire process of integrated imaging parameter weighted optimization combining neural networks and genetic algorithms in this application;

[0037] Figure 2 shows the preliminary optimization process of training and prediction based on fully connected neural networks proposed in this application;

[0038] Figure 3a is a schematic diagram of the fully connected neural network model for FCNN integrated imaging parameter prediction proposed in this application;

[0039] Figure 3b is a schematic diagram of a unit in the FCNN network structure proposed in this application;

[0040] Figure 4 is a schematic diagram of the parameter optimization process based on the multi-objective genetic algorithm proposed in this application. Specific Implementation

[0041] Various embodiments will now be described in detail, with one or more examples of these embodiments illustrated in the figures. The examples are provided for illustrative purposes and are not intended to be limiting. For example, features illustrated or described as part of one embodiment can be used in or combined with any other embodiment to produce yet another embodiment. This application is intended to include such modifications and variations.

[0042] In the following description of the accompanying drawings, the same reference numerals indicate the same or similar structures. Generally, only the differences between individual embodiments will be described. Unless otherwise expressly indicated, the description of parts or aspects of one embodiment can also be applied to corresponding parts or aspects of another embodiment.

[0043] Example 1

[0044] Referring to Figure 1, which illustrates an integrated imaging parameter optimization method combining deep learning and multi-objective genetic algorithms, characterized by the following steps:

[0045] As shown in Figure 2, the training phase of the deep learning neural network model generates random optimized combinations based on the parameter relationships of the basic integrated imaging system, and applies specific constraints. The neural network is then trained using the preprocessed dataset.

[0046] The dataset preprocessing process is shown in Figure 2, and it is characterized by the following steps:

[0047] Generate data sets freely based on the integrated imaging relation formula, with three dimensions at both ends;

[0048] The screening ensures that the data group corresponds to the real or virtual image mode of the integrated imaging, that is, the focal length of the lens in the microlens array is less than the distance between the lens array and the display screen or vice versa;

[0049] Due to imaging requirements, excessively large data sizes are limited: ensuring that resolution, depth of field, and field of view remain within reasonable limits.

[0050] Repeat the above process until a sufficient number of data sets are obtained to form a training dataset.

[0051] Advantageously, this preprocessing isolates the risk of gradient explosion during training. Training the network using this dataset is characterized by efficient training, relatively accurate model results, and close relevance to real-world scenarios.

[0052] The network structure of the constructed FCNN algorithm model. The FCNN neural network model is an end-to-end deep neural network used for back-predicting system parameters of ensemble imaging.

[0053] Preferably, the schematic diagram of the FCNN network structure is shown in Figure 3a, and the structure of each hidden layer is shown in Figure 3b. Several hidden layers are used, and the activation function is ReLU. Its structure can be represented as follows:

[0054] Input~FCC-ReLU~FCC-ReLU~…~FCC-ReLU~Output

[0055] Preferably, the root mean square error (RMSE) is used as the loss function during network training.

[0056] Preferably, the network weights are updated using the ADAM algorithm to optimize parameters.

[0057] Understandably, the neural network training process simulates the correspondence between the display parameters and system parameters of an integrated imaging system, thereby enabling the reverse prediction of design values ​​based on display requirements. To increase the efficiency of the optimization process, the model trained here has a relatively low accuracy.

[0058] Advantageously, this method utilizes neural network model prediction to greatly improve efficiency in the early stages of optimization, and can provide a rough parameter solution.

[0059] According to another preferred embodiment of this application, after obtaining a rough optimization solution through reverse prediction, an iterative process using a SPEA2-based multi-objective genetic algorithm is performed to obtain an accurate solution. A schematic diagram of the genetic algorithm optimization process is shown in Figure 4, characterized by including the following steps:

[0060] The obtained coarse solution is then subjected to a population operation, expanding the original 1×3 prediction output vector to an input suitable for the genetic algorithm: n×1×3. Simultaneously, some data fluctuation processing is applied to it.

[0061] The populationized input is fed into a genetic algorithm for a certain number of iterations to obtain the final accurate solution.

[0062] Preferably, Gaussian distribution fluctuation values ​​are added to the data during input populationization;

[0063] Preferably, the genetic fitness function is defined as a weighted sum of the root mean square errors of multiple parameters;

[0064] Advantageously, the integrated imaging parameter precision optimization method based on multi-objective genetic algorithm in this application has the advantages of being fast, efficient, and of high quality.

[0065] According to another preferred embodiment of this application, by applying the optimized integrated imaging system parameter combination to an actual display system, an integrated imaging naked-eye 3D display effect that closely matches the required display performance can be quickly and effectively achieved.

[0066] Advantageously, the integrated imaging parameter optimization method based on the fusion of deep learning and multi-objective genetic algorithm proposed in this application can quickly and efficiently obtain accurate combinations of integrated imaging system parameters that meet the needs of different application fields, thereby improving the application value of integrated imaging systems.

[0067] Compared with the prior art, this application has one or more of the following technical effects:

[0068] First, this application utilizes a trained FCNN neural network for coarse parameter estimation, thereby enabling the rapid acquisition of parameter combinations close to the optimal solution in the initial stage, and improving the optimization speed of integrated imaging system parameter design.

[0069] Secondly, this application performs a series of preprocessing steps on the randomly generated optimization results when training the neural network. The generated dataset accelerates the convergence of the loss function during training and makes the entire dataset more suitable for actual parameter combinations, effectively improving the network training efficiency and practical value.

[0070] Third, this application uses a multi-objective genetic algorithm based on SPEA2 to optimize the exact solution and populate the initial population of the coarse solution, which can significantly reduce the number of iterations, shorten the optimization speed of the integrated imaging system design parameters, and achieve a fast and efficient optimization process.

[0071] Fourth, this application defines a weighted fitness function in the multi-objective genetic algorithm process, which can change the optimization direction for different application requirements and improve the practical value of parameter optimization of integrated imaging systems.

[0072] While the foregoing describes embodiments of this application, other and further embodiments of this application may be devised without departing from the basic scope of this application, the scope of which is defined by the claims.

[0073] The above embodiments are merely preferred embodiments of this application and are not intended to limit this application. Technical features in these embodiments that do not contradict each other can be combined with each other. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithms, characterized in that... include: Based on the formula between the display device parameters and display performance parameters of the basic integrated imaging system, a training dataset is randomly generated by a multi-objective genetic algorithm. The training dataset is created by taking the display performance parameters in the dataset and the known display device parameters as inputs, and the display device parameters to be optimized as outputs. Construct a Full Connect Neural Network (FCNN) algorithm model; Train the FCNN network model and update the network weight parameters; Save the network model corresponding to the FCNN with the best training result; The target display parameters are fed into the constructed FCNN network model, and a rough system parameter value is obtained by back-prediction. The predicted values ​​are expanded into the initial population for the genetic algorithm; The exact solution was obtained by using the multi-objective SPEA2 genetic algorithm for optimization.

2. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to claim 1, characterized in that... Preliminary predictions utilize the FCNN neural network structure for inverse prediction of integrated imaging parameters.

3. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to claim 1, characterized in that... The dataset preprocessing method includes optimizing the selection of results.

4. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to claim 1, characterized in that... The neural network model used is a fully connected network model, and its structure includes... Multiple fully connected units, each of which consists of a fully connected layer and an activation layer; The output layer is an identity layer. The input consists of a multi-dimensional vector containing display parameters, known device parameters, and a number of weights. The output is a three-dimensional vector.

5. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to any one of claims 1-4, Its features are its features The training process for an FCNN network model includes the following steps: The preprocessed dataset is used as input to the FCNN network model for training; Define the root mean square error loss function and calculate the loss; Update the model weights and biases using the backpropagation algorithm; Repeat the above steps until the loss function converges.

6. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to claim 1, characterized in that... The SPEA2-based multi-objective genetic algorithm is then used to precisely optimize the integrated imaging parameters.

7. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to claim 1, characterized in that... The vector populationization process, in which the predicted output values ​​from the neural network are input into the genetic algorithm.

8. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to claim 1, characterized in that... The defined fitness function is a weighted sum of the root mean square errors of multiple display parameters.

9. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to any one of claims 1, 6-7, is characterized in that... The process of performing precise optimization using a multi-objective genetic algorithm includes the following steps: The predicted coarse solution is then subjected to a population-based operation, expanding the original 1×3 prediction output vector to an input suitable for the genetic algorithm: n×1×3. Simultaneously, some data fluctuation processing is applied to it. The populationized input is fed into a genetic algorithm, and selection, mating, and mutation operations are performed in each generation; Repeat the above steps until the required number of iterations is reached.

10. The integrated imaging parameter optimization method based on deep learning and multi-objective genetic algorithm according to claims 1-9, characterized in that: A joint optimization method combining coarse prediction using deep learning neural networks and precise optimization using multi-objective genetic algorithms.

Images