A three-dimensional model ray tracing method based on meta learning

Abstract

The application discloses a three-dimensional model light ray tracing method based on meta learning, and belongs to the technical field of three-dimensional model image processing, and comprises the following steps: step one, constructing a general model for light ray tracing; step two, training a previous model by using a stereo matching convolutional neural network; step three, jointly training the previous model to obtain a new model; and step four, finding customized parameters and a customized model of the new model meeting a specific light ray tracing effect.The application trains an old model by using a stereo matching convolutional neural network based on the construction of a general model, trains general parameters meeting a specific light tracing effect, obtains new potential general parameters through the convergence of a loss function, adapts to a specific light tracing effect, effectively trains the old model by removing 90% of previous data and 98% of previous data, reduces the training data capacity, improves the accuracy of new and old data, and thus solves the problems of long training time, large training data capacity and low light ray tracing effect accuracy of the light ray tracing technology.

Description

Technical Field

[0001] This invention relates to a ray tracing method, and more particularly to a ray tracing method for 3D models based on meta-learning, belonging to the field of 3D model image processing technology. Background Technology

[0002] Ray tracing is a method that makes objects appear three-dimensional on a two-dimensional screen. With the development of 3D imaging technology, it is widely used in architectural effect video animation, 3D games and 3D movies. It can add dynamism to images and make the weather, spring, summer, autumn and winter, sunrise and sunset in architectural 3D models, games and movies vividly displayed, especially for rendering and enhancing architectural 3D models.

[0003] Currently, although high dynamic range (HDR) architectural animations, games, and movies have very good lighting effects, they are still far from realistic lighting effects. Calculating the correct reflection and refraction angles based on the light source's position cannot completely and realistically represent the visual effects of reality. Each ray tracing operation requires a huge amount of computation, causing excessive load on graphics cards, making them unable to handle the workload. When a new ray tracing angle appears on the 3D model, that is, when the light source position is different from the previously calculated position, the model needs to be able to learn the new ray tracing angle and create new ray tracing effects for the new 3D model. In other words, while remembering the old light source position, it needs to learn new ray tracing angles. Another reason why ray tracing technology cannot realistically reflect real-world lighting effects is that it cannot superimpose color effects on light. Superimposing different colors on different light angles can make the lighting effects more realistic. In order to superimpose different colors on light from different angles, the model needs to be able to learn different effects superimposed on different ray tracing angles. While gradually deepening its learning, it should memorize old ray tracings and their corresponding light colors, and at the same time, expand to learn new ray tracing effects as its learning progresses.

[0004] Currently, traditional ray tracing effect learning methods involve retraining the model using new tracing angles. However, this method often suffers from low accuracy. Insufficient training leads to low accuracy in new ray tracing effects and poor lighting effects in the new 3D model; conversely, overtraining reduces the accuracy of old ray tracing effects and degrades the visual quality of the old 3D model. The inconsistencies between new and old data are difficult to resolve. Directly mixing and retraining convolutional neural networks results in excessively large training data volumes and excessively long training times, leading to low ray tracing efficiency and an inability to flexibly match the movement of light rays. Therefore, finding a method to reduce training data volume while achieving high-accuracy ray tracing effects has become an urgent problem to solve. Summary of the Invention

[0005] The main purpose of this invention is to solve the problems of excessively long training time, excessively large training data volume, and low accuracy of ray tracing in existing ray tracing technologies, which result in poor lighting and shadow effects in 3D models. This invention provides a ray tracing method for 3D models based on meta-learning.

[0006] The objective of this invention can be achieved by adopting the following technical solution:

[0007] A ray tracing method for 3D models based on meta-learning includes the following steps:

[0008] Step 1: Construct a general ray tracing model: Select several ray incident angles and ray colors as meta-tasks, and combine them with meta-learning methods to set meta-learning tasks. These meta-learning tasks learn a stereo matching convolutional neural network, and obtain the training parameters for ray tracing angles from the convolutional stereo matching network. and light color ;

[0009] Step 2: Train the pre-model using a stereo matching convolutional neural network: Utilize model similarity to create a stereo matching network of the same type as the stereo matching convolutional neural network, and use this model as the pre-model. Import the parameters of the stereo matching convolutional neural network into the pre-model. The pre-model uses the cross-entropy loss function to guide the parameters. and Training is then performed, followed by random sampling using cluster random sampling, removing 90% of the initial data. And 98% of the previous data And use the remaining data as memory data. and ;

[0010] Step 3: Jointly train the previous model to obtain the new model: The previous model utilizes the memorized data and And new data and Combined for joint training, the cross-entropy function guides the training of new data. and To perform model learning, on the memorized data and The learning is jointly guided by the cross-entropy loss function and the knowledge distillation loss function to obtain a new model based on the previous model.

[0011] Step 4: Discover new model parameters and customize the model to achieve specific ray tracing effects: Ray tracing angle for general training parameters. and light color Create general parameters ,use This represents a general ray tracing model with general parameters. With ray tracing angle and light color All are positively correlated, the general parameters The formula is:

[0012] ,

[0013] A dataset is defined as ray tracing data for a specific ray tracing effect. Two independent mini-blocks and from Extracting from the dataset, the meta-learner applies the following to the dataset: Training, training to adapt to specific ray tracing effects , and Relatedly, we define 'u' as the model label suitable for this specific ray tracing effect, and use a general model in... middle Predicting the binary mixture model The loss for this segment is calculated, and a personalized model adapted to this specific ray tracing effect can be trained using the above method. ,in:

[0014] ,

[0015] In the formula: This is recorded as the training and learning speed for this stage.

[0016] As a further aspect of the present invention, the stereo matching convolutional neural network is either an end-to-end stereo matching network based on convolutional neural networks or a binocular stereo matching network based on convolutional neural networks.

[0017] As a further aspect of the present invention, the loss in step four is calculated using a loss function formula, which is:

[0018] ,

[0019] In the formula: y represents the type number of the ray tracing effect. This is recorded as the loss for this stage.

[0020] As a further aspect of the present invention, the meta-learning method in step one is based on classification learning based on the effect of ray tracing. The meta-learning task based on the effect of ray tracing is divided into M tasks according to the number of classification levels of the effect of ray tracing, wherein each task contains N data samples. The loss function formula is used to calculate the loss of each data sample in each meta-learning task.

[0021] As a further aspect of the present invention, step three uses a combination of cross-entropy loss function and knowledge distillation loss function to analyze the memory data. and The guided learning method uses a comprehensive loss function, which is expressed by multiplying the cross-entropy loss function and the knowledge distillation loss function by their respective weights and then summing them. The formula for the comprehensive loss function is:

[0022] ,

[0023] In the formula: Let it be denoted as the comprehensive loss function. Let it be denoted as the cross-entropy loss function. Let this be denoted as the knowledge distillation loss function.

[0024] in, The solution formula is:

[0025] ,

[0026] In the formula: These are the potential parameters in the new data that meet specific ray tracing effects.

[0027] .

[0028] The beneficial technical effects of this invention are as follows: According to the meta-learning-based 3D model ray tracing method of this invention, by constructing a general model, an old model can be trained using a stereo matching convolutional neural network. Based on the input of general parameters, general parameters that can satisfy specific ray tracing effects are trained. Furthermore, new potential general parameters are obtained through the convergence of the loss function, which can adapt to specific ray tracing effects and meet the requirements of ray tracing expression. This is achieved by removing 90% of the previous data. And 98% of the previous data And use the remaining data as memory data. and It can effectively train old models, fine-tune and memorize the parameters in the old models, reduce the amount of training data, shorten the training time of ray tracing technology, and improve the accuracy of new and old data. This solves the shortcomings of ray tracing technology, such as long training time, large training data volume, and low accuracy of ray tracing, which leads to poor lighting and shadow effects in 3D models. By solving the ray tracing parameters required for specific ray tracing effects, it can meet the lighting and shadow effects in specific scenes, so that 3D building models can be displayed more vividly in specific scenes and improve the realism of 3D building models. Attached Figure Description

[0029] Figure 1 This is a flowchart of the overall method according to the present invention. Detailed Implementation

[0030] To enable those skilled in the art to understand the technical solution of the present invention more clearly, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto. Example

[0031] This embodiment sets a set of previous data. and This set of previous data and It consists of ray tracing angles and light colors for sunny and cloudy days, and then a new set of data is set. This embodiment is designed to improve the accuracy of ray tracing effects for both new and old data.

[0032] like Figure 1 As shown, the method implemented in this embodiment is as follows:

[0033] Step 1: Construct a general ray tracing model: Select several ray incident angles and ray colors as meta-tasks. Combine this with meta-learning methods to set meta-learning tasks. For example, select the first set of ray incident angles and colors (ray incident height angle of 30°, incident direction of 10° south of west, ray color of yellow) as the first task; select the second set of ray incident angles and colors (ray incident height angle of 45°, incident direction of 20° south of west, ray color of green) as the second task; and select the third set of ray incident angles and colors (ray incident height angle of 60°, incident direction of 30° south of west) as the third task. These meta-learning tasks learn a stereo matching convolutional neural network. The training parameters for ray tracing angles are obtained from the convolutional stereo matching network. and light color Stereo matching convolutional neural networks are either end-to-end stereo matching networks based on convolutional neural networks or binocular stereo matching networks based on convolutional neural networks.

[0034] Step 2: Train the pre-model using a stereo matching convolutional neural network: Utilize model similarity to create a stereo matching network of the same type as the stereo matching convolutional neural network, and use this model as the pre-model. Import the parameters of the stereo matching convolutional neural network into the pre-model. The pre-model uses the cross-entropy loss function to guide the parameters. and Training is then performed, followed by random sampling using cluster random sampling, removing 90% of the initial data. And 98% of the previous data And use the remaining data as memory data. and ;

[0035] Step 3: Jointly train the previous model to obtain the new model: The previous model utilizes the memorized data and And new data and Combined for joint training, the cross-entropy function guides the training of new data. and To perform model learning, on the memorized data and The learning is jointly guided by the cross-entropy loss function and the knowledge distillation loss function to obtain a new model based on the previous model.

[0036] Step 4: Discover new model parameters and customize the model to achieve specific ray tracing effects: Ray tracing angle for general training parameters. and light color Create general parameters ,use This represents a general ray tracing model with general parameters. With ray tracing angle and light color All are positively correlated, the general parameters The formula is:

[0037] ,

[0038] Among them, the ray tracing angle It is a comprehensive parameter that takes into account both the incident elevation angle and the direction angle of the light. The formula for the ray tracing angle is:

[0039] ,

[0040] In the formula: It is denoted as the angle of incidence of the light ray. It is denoted as the angle of incidence of the light ray.

[0041] The purpose of this setting is to place the orientation angle in the integer part of the ray tracing angle and the ray incident azimuth angle in the unit part of the ray tracing angle, making it easier to observe the parameters intuitively. At the same time, considering both factors, it simplifies the calculation steps and reduces computational complexity, thus helping to improve the calculation speed. Through the above general parameters... The settings can comprehensively utilize the ray tracing angle and ray color to calculate a specific value to replace the ray incident elevation angle, direction angle and ray color. This makes it easier to select the values ​​suitable for a specific ray tracing effect from the ray incident elevation angle, direction angle and ray color, simplifying the complexity of the calculation and helping to improve the calculation speed.

[0042] A dataset is defined as ray tracing data for a specific ray tracing effect. Two independent mini-blocks and from Extracting from the dataset, the meta-learner applies the following to the dataset: Training, training to adapt to specific ray tracing effects , and Relatedly, we define 'u' as the model label suitable for this specific ray tracing effect, and use a general model in... middle Predicting the binary mixture model The loss for this segment is calculated, and a personalized model adapted to this specific ray tracing effect can be trained using the above method. ,in:

[0043] ,

[0044] In the formula: This is recorded as the training and learning speed for this stage.

[0045] The loss in this stage, as described in step four, is calculated using a loss function formula, which is:

[0046] ,

[0047] In the formula: y represents the type number of the ray tracing effect. This is recorded as the loss for this stage.

[0048] The meta-learning method described in step one is based on classification learning based on the effect of ray tracing. The meta-learning task based on the effect of ray tracing is divided into M tasks according to the number of classification levels of the ray tracing effect. Each task contains N data samples. The loss function formula is used to calculate the loss of each data sample in each meta-learning task.

[0049] Step three uses a combination of cross-entropy loss function and knowledge distillation loss function to analyze the memorized data. and The guided learning method uses a comprehensive loss function, which is expressed by multiplying the cross-entropy loss function and the knowledge distillation loss function by their respective weights and then summing them. The formula for the comprehensive loss function is:

[0050] ,

[0051] In the formula: Let it be denoted as the comprehensive loss function. Let it be denoted as the cross-entropy loss function. Let this be denoted as the knowledge distillation loss function.

[0052] By setting a comprehensive loss function, we can give more consideration to the cross-entropy loss function while also taking into account the knowledge distillation loss function, thus making the loss function compatible and comprehensively reactive.

[0053] in, The solution formula is:

[0054] ,

[0055] In the formula: These are the potential parameters in the new data that meet specific ray tracing effects.

[0056] .

[0057] In summary, in this embodiment, the meta-learning-based 3D model ray tracing method can train an old model using a stereo matching convolutional neural network by constructing a general model. Based on the general parameter input, it trains general parameters that can satisfy specific ray tracing effects. Furthermore, new potential general parameters are obtained through the convergence of the loss function, which can adapt to specific ray tracing effects and meet the requirements of ray tracing expression. This is achieved by removing 90% of the previous data. And 98% of the previous data And use the remaining data as memory data. and It can effectively train old models, fine-tune and memorize the parameters in the old models, reduce the amount of training data, shorten the training time of ray tracing technology, and improve the accuracy of new and old data. This solves the shortcomings of ray tracing technology, such as long training time, large training data volume, and low accuracy of ray tracing, which leads to poor lighting and shadow effects in 3D models. By solving the ray tracing parameters required for specific ray tracing effects, it can meet the lighting and shadow effects in specific scenes, so that 3D building models can be displayed more vividly in specific scenes and improve the realism of 3D building models.

[0058] The above description is merely a further embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A ray tracing method for 3D models based on meta-learning, characterized in that, Includes the following steps: Step 1: Construct a general ray tracing model: Select several ray incident angles and ray colors as meta-tasks, and combine meta-learning methods to set meta-learning tasks. Each meta-learning task trains a stereo matching network based on a convolutional neural network, and the angle parameters used for ray tracing are obtained based on the training results of the stereo matching network. and light color ; Step 2: Train the pre-model using a stereo matching convolutional neural network: Utilize model similarity to create a stereo matching network of the same type as the stereo matching convolutional neural network, and use this same type of stereo matching network as the pre-model. Import the parameters of the stereo matching convolutional neural network into the pre-model, and use the cross-entropy loss function to guide the parameters. and Training is then performed, followed by random sampling using cluster random sampling, removing 90% of the initial data. And 98% of the previous data And use the remaining data as memory data. and ; Step 3: Jointly train the previous model to obtain the new model: The previous model utilizes the memorized data and And new data and Combined for joint training, the cross-entropy function guides the training of new data. and To perform model learning, on the memorized data and Learning is jointly guided by cross-entropy loss function and knowledge distillation loss function to obtain a new model based on the previous model; Step 4: Discover new model parameters and customize the model to achieve specific ray tracing effects: Ray tracing angle for general training parameters. and light color Create general parameters ,use This represents a general ray tracing model with general parameters. With ray tracing angle and light color All are positively correlated, the general parameters The formula is: , A dataset is defined as ray tracing data for a specific ray tracing effect. Two independent mini-blocks and from Extracting from the dataset, the meta-learner applies the following to the dataset: Training, training to adapt to specific ray tracing effects , and Relatedly, 'u' is defined as the model label suitable for this specific ray tracing effect, using a general ray tracing model. exist The binary mixture model was predicted. The loss at this stage is calculated, and a personalized model adapted to this specific ray tracing effect can be trained using the above method. ,in: ， In the formula: This is recorded as the training and learning speed at this stage. This represents the losses incurred during this phase.

2. The ray tracing method for 3D models based on meta-learning as described in claim 1, characterized in that, The stereo matching convolutional neural network is either an end-to-end stereo matching network based on convolutional neural networks or a binocular stereo matching network based on convolutional neural networks.

3. The ray tracing method for 3D models based on meta-learning as described in claim 1, characterized in that, The loss in this stage, as described in step four, is calculated using a loss function formula, which is: ， In the formula: y represents the type number of the ray tracing effect.

4. The ray tracing method for 3D models based on meta-learning as described in claim 3, characterized in that, The meta-learning method described in step one is based on classification learning based on the effect of ray tracing. The meta-learning task based on the effect of ray tracing is divided into M tasks according to the number of classification levels of the ray tracing effect. Each task contains N data samples. The loss function formula is used to calculate the loss of each data sample in each meta-learning task.

5. The ray tracing method for 3D models based on meta-learning as described in claim 1, characterized in that, In step three, the cross-entropy loss function and the knowledge distillation loss function are used in combination to analyze the memorized data. and The guided learning method uses a comprehensive loss function, which is expressed by multiplying the cross-entropy loss function and the knowledge distillation loss function by their respective weights and then summing them. The formula for the comprehensive loss function is: ， In the formula: Let it be denoted as the comprehensive loss function. Let it be denoted as the cross-entropy loss function. Let this be denoted as the knowledge distillation loss function. in, The solution formula is: ， In the formula: These are denoted as potential parameters in the new data that meet specific ray tracing effects.

Images