Simulation game three-dimensional model establishing method

Abstract

The invention discloses a simulation game three-dimensional model building method, and relates to a three-dimensional model building method. The invention aims to solve the problem that in the scene conversion process of a game based on three-dimensional modeling, the situation of insufficient brightness of a figure image and an environment image often occurs, especially a three-dimensional model scene with rainwater, and the game experience feeling of a user is seriously influenced. The method comprises the following steps: step 1, brightening a game scene conversion image with a three-dimensional model; step 2, carrying out rain removal processing on the three-dimensional model game scene with the rain; and step 3, outputting the three-dimensional model game scene subjected to brightening and rain removal processing. The invention belongs to the technical field of online games.

Description

Technical Field

[0001] The invention relates to a three-dimensional model building method, belonging to the technical field of online games. Background Art

[0002] Computer games are a new form of entertainment that uses computers as operating platforms and is realized through human-computer interaction. It can reflect the current high level of computer technology. With the popularization of the Internet, computer network games have become popular. Computer network games are also called Internet games. They use the Internet as a transmission medium, game operator servers and user computers as processing terminals, and game client software as an information interaction window. They aim to achieve entertainment, leisure, communication and virtual achievements, and have sustainability and individual multiplayer online experience. In recent years, the types of Internet games and Internet game operators have continued to increase. Traditional two-dimensional scene games can no longer meet the needs of the majority of game enthusiasts. Three-dimensional scene simulation games continue to appear and have almost replaced traditional two-dimensional scene games; however, in the process of scene conversion in games based on three-dimensional modeling, the character images and environmental images often have insufficient brightness, especially in three-dimensional model scenes with rain, which seriously affects the user's gaming experience. Summary of the invention

[0003] The present invention aims to solve the problem that in the process of scene conversion in games based on 3D modeling, character images and environmental images often have insufficient brightness, especially in 3D model scenes with rain, which seriously affects the user's gaming experience. A method for establishing a 3D model for a simulation game is proposed.

[0004] The technical solution adopted by the present invention to solve the above-mentioned problem is: the steps of the present invention include:

[0005] Step 1, brightening the game scene conversion image with the three-dimensional model;

[0006] Step 2: Remove rain from the 3D model game scene with rain;

[0007] Step 3: Output the brightened and rain-removed 3D model game scene.

[0008] Furthermore, step 1 specifically includes:

[0009] Step 101, obtaining a dataset image;

[0010] Step 102: Dataset image enhancement;

[0011] Step 103: training a brightness enhancement model based on the LIME method;

[0012] Step 104: Model evaluation and optimization.

[0013] Furthermore, step 101 specifically includes: using automated web crawler technology to collect scene images whose details are difficult to identify; and storing all images in a data processing center after collection for subsequent image preprocessing.

[0014] Furthermore, for the collected scene image data with missing details, the following preprocessing operations are performed to enhance the diversity of the dataset and adapt to the model input requirements:

[0015] Step 1021, rotation and mirroring: the image data is first randomly rotated and mirrored; the purpose of this operation is to increase the diversity of the data set by simulating different observation angles, thereby improving the robustness and generalization ability of the model in practical applications;

[0016] Step 1022, random cropping: Randomly crop 256x256 pixel image blocks from the original coral image. This step ensures that all image blocks have a uniform size, which is suitable as the input of the deep learning model. This operation also helps the model learn to recognize and process images from local features, enhancing the model's ability to process various parts of the coral image.

[0017] Further, step 103 specifically includes: the training of the brightness enhancement model is achieved by combining the LIME method with the deep learning technology to ensure that the final model can effectively enhance the brightness and clarity of the scene image. The specific steps are as follows:

[0018] Step 1031: Model architecture design;

[0019] Design a deep learning model architecture based on a convolutional neural network. The model structure includes multiple convolutional layers, activation layers, and pooling layers. Each layer is designed to extract and process specific features of the image. The input layer of the model accepts a preprocessed scene image, and the output layer generates a brightness-enhanced scene image.

[0020] Step 1032: data preparation and preprocessing;

[0021] Before model training, necessary preprocessing is performed on the collected blurred scene images, using methods such as Gaussian blur and bilateral filtering to remove image noise while retaining important edge information;

[0022] Step 1033, training the illumination map estimation network;

[0023] A deep learning method based on Retinex theory is used. The training process is designed to enable the model to accurately learn and predict the illumination distribution under low-light conditions, thereby effectively enhancing the brightness of scene images.

[0024] The specific formula of the established Retinex theory is as follows:

[0025] I(x,y)=R(x,y)·L(x,y)

[0026] Where I(x,y) represents the pixel value of the input scene image at position (x,y); R(x,y) represents the pixel value of the reflectivity map of the three-dimensional model in the scene at position (x,y); L(x,y) represents the pixel value of the three-dimensional model map in the scene at position (x,y);

[0027] In order to train the model to accurately estimate illumination and reflectance, a joint loss function is used to minimize the error between the predicted value and the true value. This loss function includes two components: illumination estimation and reflectance estimation loss. By minimizing the error between the predicted value and the true value, the adaptability and accuracy of the model under different lighting conditions are improved.

[0028] The joint loss function is defined as:

[0029]

[0030] Among them, α and β are weight parameters used to balance the importance of illumination loss and reflectivity loss; is the loss function of illumination estimation; is the loss function of reflectivity estimation;

[0031] During the training process, the model uses the estimated light distribution to enhance the brightness of low-light coral images. By learning the characteristics of coral images under normal light conditions, the model accurately reconstructs the light map under low light conditions, improving the brightness and contrast of coral images that originally lack details;

[0032] Step 1034: brightness adjustment and optimization;

[0033] After the illumination map is estimated, histogram equalization and Gamma algorithm are applied to enhance the previously estimated illumination map to adjust the brightness and contrast of the image;

[0034] Step 1035: image fusion and output;

[0035] The enhanced scene image is fused with the original scene image with color distortion and missing details. Laplace pyramid fusion and band fusion-based image fusion technology are used to ensure that the two are seamlessly combined at the pixel level. This step accurately maintains the details of the original coral image while introducing the brightness advantage of the enhanced image through multi-scale decomposition and reconstruction. The fused image is significantly better than the original image in visual effect. It is not only brighter and clearer, but also maintains natural color expression.

[0036] Furthermore, step 104 specifically includes:

[0037] The improved structural similarity index method is used to evaluate the scene image processed by the brightness enhancement network model. The brightness adjustment factor B and the contrast adjustment factor C are added to the SSIM formula. The improved SSIM calculation formula is:

[0038]

[0039] Among them, μ x , μ y Respectively represent the brightness mean of scene images X and Y, reflecting the average light intensity of the 3D model in the scene; σ x , σ y They represent the variance of the scene image × and Y, respectively, representing the degree of illumination change in the scene; σ xy represents the covariance of scene images × and Y, describing the common trend of brightness changes of the two images; c1 and c2 are small constants used to ensure the stability of calculation and avoid the situation where the denominator is zero; B = |μ y -μ x | represents the brightness adjustment factor, reflecting the change in average light intensity during the enhancement process C = |μ y -μ x | represents the contrast adjustment factor, which quantifies the change in image contrast during the enhancement process;

[0040] For each pair of original low-light distorted and blurred scene images and visually perfect scene images in the database, the SSIM value is calculated using the processed detail-enhanced image and the perfect image to quantitatively evaluate the enhancement effect of the model; the SSIM value ranges from -1 to 1, where 1 indicates complete similarity; a high SSIM value indicates that the model effectively enhances the brightness and contrast of the image while maintaining the natural colors and details of the image; finally, according to the SSIM evaluation results, parameter adjustment and algorithm iteration are performed; including adjusting network parameters such as learning rate, number of layers, and filter size to optimize model performance. In addition, the algorithm is iteratively optimized, including adjusting the calculation method of brightness and contrast adjustment factors to finely control the improvement of scene image quality.

[0041] Furthermore, step 2 specifically includes:

[0042] Step 201, generating and screening rainy images;

[0043] Step 202, RGB image deraining model;

[0044] Step 203: Introduce global semantic guidance as additional supervision.

[0045] Furthermore, step 201 specifically includes:

[0046] Step 2011, generating rain streaks through a variational inference framework;

[0047] The principle of the rain streak synthesis network structure based on the variational inference framework is: (1)

[0048] o=r+b. (1),

[0049] Among them, o is a single rainy image, r is a rain pattern image, and b is a clean background image without rain;

[0050] The rain pattern layer r is modeled as the following generator form:

[0051] r=G θ (z).(2),

[0052] Among them, z is a hidden variable, and θ represents the parameters of the generator;

[0053] In order to ensure the isotropy of rainfall factors, the network assumes that the prior distribution of the latent variable z follows a normal distribution with a mean of 0 and a covariance of the identity matrix, that is;

[0054] z~N(z|0,I).(3),

[0055] Generate different rain patterns by adjusting the specific z dimension;

[0056] Background modeling;

[0057] Embed x into the Gaussian prior distribution of b;

[0058]

[0059] in, is a hyperparameter that measures the similarity between x and b;

[0060] According to the above formula, the probability distribution of the rain map o is:

[0061] o~p θ (o|z,b). (5),

[0062] Among them, p θ (·) indicates that the distribution also depends on the parameters θ of the generator;

[0063] The task of generating rain images is transformed into the task of learning the general distribution p(o), which is expressed as follows:

[0064] p(o)=∫∫p θ (o|z,b)p(z)p(b)dzdb. (6),

[0065] Where p(z) and p(b) represent the prior distribution of z and b respectively;

[0066] Hidden distribution learning based on variational inference framework;

[0067] Using DIFFNet model to estimate monocular depth of RGB image I RGB After obtaining d(x), a hyperparameter β is used to adjust the required scattering intensity, and the transmission rate t(x) is obtained according to the following formula.

[0068] t(x)=e -β×d(x) . (7),

[0069] The hyperparameter A is used to adjust the color and intensity of the foggy image, and the foggy image I is obtained by the following formula: haze (x);

[0070] I haze (x) = I RGB (x)×t(x)+A×(1-t(x)). (8),

[0071] The screen blending model (SMB) is used to synthesize rain streaks and background images. The formula is as follows:

[0072] R=B+SB⊙S. (9),

[0073] Among them, R represents the synthesized rain image, B represents the rain-free background image, and S represents the rain pattern image;

[0074] The dataset was cropped into 256×256 images and the VRGNet model was used to synthesize rain images with different directions, scales, and thicknesses. Finally, the above-mentioned ASM model was used to synthesize rain and fog, and a triplet was constructed for subsequent work.

[0075] Furthermore, step 202 specifically includes:

[0076] Parallel attention module; contains a gated attention, a channel attention and pixel attention; batch normalizes the input features before performing parallel attention-based feature extraction;

[0077] Gated attention can initially extract pixel-level feature information. It consists of a feature extraction branch and a gating branch. The feature extraction branch includes convolutions with kernel sizes of 1×1 and 3×3, and the gating branch includes 1×1 convolution and Sigmoid activation function.

[0078] Channel attention can effectively extract global information across channels. By assigning different importance to the features of different channels, the model can focus more on features that are more useful for the current deraining task. It first uses a global average pooling operation to extract the global spatial information of each channel, then uses point-by-point convolution to learn the complex dependencies between channels, and uses the GELU activation function to increase nonlinearity. The Sigmoid activation function generates an attention score for each channel to ensure that the attention score is between 0 and 1. Finally, the attention score of each channel is multiplied by the corresponding channel of the original feature map to obtain a weighted feature map. This strengthens important feature channels and suppresses unimportant channels.

[0079] Pixel attention can extract global pixel-gated features. It focuses more on the importance of each pixel in the feature map, allowing the model to weight features at a finer level, thereby more accurately focusing on local features that are more important for the current deraining task. It first uses point-by-point convolution and GELU activation functions to learn from the input feature map and generate the corresponding pixel-level attention map. Then, the Sigmoid activation function is used to ensure that the attention score is between 0 and 1, and finally the original feature map is subjected to feature weighting operations.

[0080] The three different attention results are connected along the channel dimension, and the dimension of the connected feature channel is reduced to the same dimension as the input using point-by-point convolution-GELU-point-by-point convolution, and the output feature map is summed up using skip connection;

[0081] Optimal selection of sparse attention module; for a layer-normalized feature map First, point-by-point convolution with a kernel size of 1×1 is used to aggregate pixel-level cross-channel features, and depth-wise convolution with a kernel size of 3×3 is used to encode channel-level spatial information, thereby obtaining the query (Q), key (K), and value (V) mapped according to the feature map X;

[0082]

[0083] in represents the depthwise convolution, represents point-wise convolution;

[0084] The original self-attention mechanism calculates the similarity between each token and other tokens to obtain an attention map;

[0085] The optimal selection mechanism is introduced to perform feature selection. After obtaining the query (Q), key (K) and value (V), the similarity between all queries and keys is calculated by performing a dot product operation on Q and K to obtain the attention map. Only the largest k self-attention score values ​​are retained, and the other self-attention score values ​​are set to negative infinity; finally, the softmax function is used to normalize the attention scores to obtain the attention weights;

[0086] Multiply the attention weight by the value (V) to calculate the weighted sum of self-attention and get the self-attention output for each position, which is expressed as follows:

[0087]

[0088] Mixed-scale feedforward networks;

[0089] For an input feature map After layer normalization, 1×1 convolution is used to expand the number of channels, and then they are sent to two parallel multi-scale deep convolution paths, and 3×3 and 5×5 deep convolutions are used to enhance multi-scale local information extraction, respectively, which is expressed as:

[0090]

[0091] Among them, f 1×1 represents 1×1 convolution, f 3×3 and f 5×5 They represent 3×3 and 5×5 depth convolutions respectively, and concat(·) represents cross-channel connection;

[0092] Attention-enhanced fusion network;

[0093] For the features F from RGB images and infrared images RGB and F IR First, the global spatial information is integrated into the corresponding channel descriptor through an adaptive mean pooling operation; in order to thoroughly capture the dependencies between different channels, a linear layer is used for mapping, and then GELU activation is used. This process generates the RGB image feature weight W RGB and infrared image feature weight W IR , expressed as:

[0094]

[0095] Among them, f GAP represents global mean pooling, f MLP represents a multi-layer perceptron;

[0096] A correlation attention matrix A is used to weigh the features F from the RGB image and the infrared image. VIS and F IR the relationship between;

[0097]

[0098] Where μ(·) represents the Sigmoid activation function; the coefficient c represents the number of channels and acts as a proportional factor for attention enhancement; It means element-wise multiplication;

[0099] In the shared feature enhancement branch, we first combine the relevant attention matrix A with the RGB feature F RGB and infrared signature F IR Multiply by channel to get the initial fusion features;

[0100] The fused features are processed through the cross-channel fusion enhancement module:

[0101]

[0102] Among them, ⊙ represents channel-wise multiplication, concat(·) represents cross-channel concatenation, and f 7×7 Represents a depthwise convolution with a convolution kernel size of 7×7;

[0103] Simply perform complementary feature attention enhancement, recorded as:

[0104]

[0105] The spatial attention mechanism is used to aggregate the enhanced RGB and infrared features as output, and the fused features are obtained in the following way:

[0106]

[0107] Among them, f 1×1 Indicates that the convolution kernel size is 1×1 convolution;

[0108] Will get Split to obtain the fused RGB image features and infrared image features Add them together to get the fusion feature F fuse ;

[0109]

[0110] Among them, split(·) means splitting by channel;

[0111] Attention-guided expert networks;

[0112] The attention-guided expert network contains a total of 8 expert networks, namely: an average pooling layer with a receptive field of 3×3, a separable convolution layer with convolution kernel sizes of 1×1, 3×3, 5×5, and 7×7, and an expanded convolution layer with convolution kernels of 3×3, 5×5, and 7×7;

[0113] For the input feature map First, use the adaptive pooling layer to get the descriptor corresponding to each channel

[0114]

[0115] Among them, X(i, j, k) represents the pixel value at (i, j) of the kth channel of the feature map x;

[0116] The learnable weight matrices W1 and W2 are then used to obtain the coefficients T of each expert network:

[0117] T=W2σ(W1z),

[0118]

[0119] Where σ(·) represents the ReLU activation function, f 1×1 represents 1×1 convolution, represents the expert network operation, [·] represents the cross-channel connection;

[0120] The Charbonnier Loss loss function is used to train the IRDeNet network. The formula of the loss function is as follows:

[0121]

[0122] Among them, I gt Indicates a true rainless image, I derain Represents the rain-free image generated by IRDeNet.

[0123] Furthermore, step 203 specifically includes: respectively performing the real rain-free image I gt And the rain-free image after rain removal I Derain Use the pre-trained SAM model to segment the image, get their corresponding mask maps, and then construct a loss function to measure the gap between the two; the constructed loss function (seg loss) consists of focal loss and dice loss:

[0124]

[0125] Among them, a and β are adjustable hyperparameters;

[0126] Focal loss is used to solve the problem of imbalanced sample categories, which is defined as:

[0127]

[0128] Among them, p t It is expressed as:

[0129]

[0130] Dice loss is used to evaluate the similarity between two samples. It measures the performance of the segmentation model by comparing the similarity between the predicted results and the true labels. It is defined as:

[0131]

[0132] Among them, y i Indicates the value predicted by the model, ranging from 0 to 1, t i represents the target value;

[0133] The overall loss function of the IRDeNet network is expressed as:

[0134]

[0135] The beneficial effects of the present invention are:

[0136] 1. The present invention solves the problem that in the process of scene conversion in games based on 3D modeling, the character images and environmental images often have insufficient brightness, especially in 3D model scenes with rain, which seriously affects the user's gaming experience;

[0137] 2. The present invention effectively reduces high-frequency information in the image, thereby smoothing the image and significantly reducing visual noise. In addition, in order to better preserve the edge information in the scene image, the present invention also uses edge-preserving filters, such as bilateral filtering and non-local mean filtering, to perform image clarity processing, ensuring the efficiency and accuracy of the entire image processing process;

[0138] 3. The present invention more realistically reflects the natural color of the 3D model in the scene image by optimizing the contrast and detail of the image, making the texture and structure of the 3D model more obvious and clear, and significantly improving the overall visual quality of the image;

[0139] 4. The present invention can also effectively remove rain streaks in rainy scene pictures, ensuring the brightness and clarity of the scene after scene conversion, giving users a real and good experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0140] Figure 1 This is the structure diagram of the parallel attention module;

[0141] Figure 2 This is the optimal choice of sparse attention module structure diagram;

[0142] Figure 3 It is a diagram of the mixed-scale feedforward network structure;

[0143] Figure 4 This is the structure diagram of the attention enhancement fusion network;

[0144] Figure 5 This is the structure diagram of the attention-guided expert network. DETAILED DESCRIPTION

[0145] Specific implementation method 1: Figures 1 to 5 As shown, a method for establishing a three-dimensional model of a simulation game, the specific steps include:

[0146] Step 1, brightening the game scene conversion image with the three-dimensional model;

[0147] Step 2: Remove rain from the 3D model game scene with rain;

[0148] Step 3: Output the brightened and rain-removed 3D model game scene.

[0149] Specific implementation method 2: Figures 1 to 5 As shown, step 1 specifically includes:

[0150] Step 101, obtaining a dataset image;

[0151] Step 102: Dataset image enhancement;

[0152] Step 103: training a brightness enhancement model based on the LIME method;

[0153] Step 104: Model evaluation and optimization.

[0154] Specific implementation method three: Figures 1 to 5 As shown, step 101 specifically includes: using automated web crawler technology to collect scene images whose details are difficult to identify; all images are stored in a data processing center after collection for subsequent image preprocessing.

[0155] Specific implementation method four: Figures 1 to 5 As shown, step 102 specifically includes: for the collected scene image data with missing details, performing the following preprocessing operations to enhance the diversity of the data set and adapt to the model input requirements:

[0156] Step 1021, rotation and mirroring: the image data is first randomly rotated and mirrored; the purpose of this operation is to increase the diversity of the data set by simulating different observation angles, thereby improving the robustness and generalization ability of the model in practical applications;

[0157] Step 1022, random cropping: Randomly crop 256x256 pixel image blocks from the original coral image. This step ensures that all image blocks have a uniform size, which is suitable as the input of the deep learning model. This operation also helps the model learn to recognize and process images from local features, enhancing the model's ability to process various parts of the coral image.

[0158] Specific implementation method five: Figures 1 to 5 As shown, step 103 specifically includes: the training of the brightness enhancement model is achieved by combining the LIME method with the deep learning technology to ensure that the final model can effectively enhance the brightness and clarity of the scene image. The specific steps are as follows:

[0159] Step 1031: Model architecture design;

[0160] Design a deep learning model architecture based on a convolutional neural network. The model structure includes multiple convolutional layers, activation layers, and pooling layers. Each layer is designed to extract and process specific features of the image. The input layer of the model accepts a preprocessed scene image, and the output layer generates a brightness-enhanced scene image.

[0161] Step 1032: data preparation and preprocessing;

[0162] Before model training, necessary preprocessing is performed on the collected blurred scene images, using methods such as Gaussian blur and bilateral filtering to remove image noise while retaining important edge information;

[0163] Step 1033, training the illumination map estimation network;

[0164] A deep learning method based on Retinex theory is used. The training process is designed to enable the model to accurately learn and predict the illumination distribution under low-light conditions, thereby effectively enhancing the brightness of scene images.

[0165] The specific formula of the established Retinex theory is as follows:

[0166] I(x,y)=R(x,y)·L(x,y)

[0167] Where I(x,y) represents the pixel value of the input scene image at position (x,y); R(x,y) represents the pixel value of the reflectivity map of the three-dimensional model in the scene at position (x,y); L(x,y) represents the pixel value of the three-dimensional model map in the scene at position (x,y);

[0168] In order to train the model to accurately estimate illumination and reflectance, a joint loss function is used to minimize the error between the predicted value and the true value. This loss function includes two components: illumination estimation and reflectance estimation loss. By minimizing the error between the predicted value and the true value, the adaptability and accuracy of the model under different lighting conditions are improved.

[0169] The joint loss function is defined as:

[0170]

[0171] Among them, α and β are weight parameters used to balance the importance of illumination loss and reflectivity loss; is the loss function of illumination estimation; is the loss function of reflectivity estimation;

[0172] During the training process, the model uses the estimated light distribution to enhance the brightness of low-light coral images. By learning the characteristics of coral images under normal light conditions, the model accurately reconstructs the light map under low light conditions, improving the brightness and contrast of coral images that originally lack details;

[0173] Step 1034: brightness adjustment and optimization;

[0174] After the illumination map is estimated, histogram equalization and Gamma algorithm are applied to enhance the previously estimated illumination map to adjust the brightness and contrast of the image;

[0175] Step 1035: image fusion and output;

[0176] The enhanced scene image is fused with the original scene image with color distortion and missing details. Laplace pyramid fusion and band fusion-based image fusion technology are used to ensure that the two are seamlessly combined at the pixel level. This step accurately maintains the details of the original coral image while introducing the brightness advantage of the enhanced image through multi-scale decomposition and reconstruction. The fused image is significantly better than the original image in visual effect. It is not only brighter and clearer, but also maintains natural color expression.

[0177] Specific implementation method six: Figures 1 to 5 As shown, step 104 specifically includes:

[0178] The improved structural similarity index method is used to evaluate the scene image processed by the brightness enhancement network model. The brightness adjustment factor B and the contrast adjustment factor C are added to the SSIM formula. The improved SSIM calculation formula is:

[0179]

[0180] Among them, μ x , μ y Respectively represent the brightness mean of scene images X and Y, reflecting the average light intensity of the 3D model in the scene; σ x , σ y They represent the variance of scene images X and Y, respectively, representing the degree of illumination change in the scene; σ xy Represents the covariance of scene images X and Y, describing the common trend of brightness changes of the two images; c1 and c2 are small constants used to ensure the stability of the calculation and avoid the situation where the denominator is zero. B = |μy -μ x | represents the brightness adjustment factor, reflecting the change in average light intensity during the enhancement process C = |μ y -μ x | represents the contrast adjustment factor, which quantifies the change in image contrast during the enhancement process;

[0181] For each pair of original low-light distorted and blurred scene images and visually perfect scene images in the database, the SSIM value is calculated using the processed detail-enhanced image and the perfect image to quantitatively evaluate the enhancement effect of the model; the SSIM value ranges from -1 to 1, where 1 indicates complete similarity; a high SSIM value indicates that the model effectively enhances the brightness and contrast of the image while maintaining the natural colors and details of the image; finally, according to the SSIM evaluation results, parameter adjustment and algorithm iteration are performed; including adjusting network parameters such as learning rate, number of layers, and filter size to optimize model performance. In addition, the algorithm is iteratively optimized, including adjusting the calculation method of brightness and contrast adjustment factors to finely control the improvement of scene image quality.

[0182] Specific implementation method seven: Figures 1 to 5 As shown, step 2 specifically includes:

[0183] Step 201, generating and screening rainy images;

[0184] Step 202, RGB image deraining model;

[0185] Step 203: Introduce global semantic guidance as additional supervision.

[0186] Specific implementation method eight: Figures 1 to 5 As shown, step 201 specifically includes:

[0187] Step 2011, generating rain streaks through a variational inference framework;

[0188] The principle of the rain streak synthesis network structure based on the variational inference framework is: (1)

[0189] o=r+b. (1),

[0190] Among them, o is a single rainy image, r is a rain pattern image, and b is a clean background image without rain;

[0191] The rain pattern layer r is modeled as the following generator form:

[0192] r=G θ (z). (2),

[0193] Among them, z is a hidden variable, and θ represents the parameters of the generator;

[0194] In order to ensure the isotropy of rainfall factors, the network assumes that the prior distribution of the latent variable z follows a normal distribution with a mean of 0 and a covariance of the identity matrix, that is;

[0195] z~N(z|0,I).(3),

[0196] Generate different rain patterns by adjusting the specific z dimension;

[0197] Background modeling;

[0198] Embed the remainder into the Gaussian prior distribution of b;

[0199]

[0200] in, is a hyperparameter that measures the similarity between x and b;

[0201] According to the above formula, the probability distribution of the rain map o is:

[0202] o~p θ (o|z,b). (5),

[0203] Among them, p θ (·) indicates that the distribution also depends on the parameters θ of the generator;

[0204] The task of generating rain images is transformed into the task of learning the general distribution p(o), which is expressed as follows:

[0205] p(o)=∫∫P θ (o|z,b)p(z)p(b)dzdb. (6),

[0206] Where p(z) and p(b) represent the prior distribution of z and b respectively;

[0207] Hidden distribution learning based on variational inference framework;

[0208] Using DIFFNet model to estimate monocular depth of RGB image I RGB After obtaining d(x), a hyperparameter β is used to adjust the required scattering intensity, and the transmission rate t(x) is obtained according to the following formula;

[0209] t(x)=e -β×d(x) . (7),

[0210] The hyperparameter A is used to adjust the color and intensity of the foggy image, and the foggy image I is obtained by the following formula: haze (x);

[0211] I haze (x) = I RGB(x)×t(x)+A×(1-t(x)). (8),

[0212] The screen blending model (SMB) is used to synthesize rain streaks and background images. The formula is as follows:

[0213] R=B+SB⊙S. (9),

[0214] Among them, R represents the synthesized rain image, B represents the rain-free background image, and S represents the rain pattern image;

[0215] The dataset was cropped into 256×256 images and the VRGNet model was used to synthesize rain images with different directions, scales, and thicknesses. Finally, the above-mentioned ASM model was used to synthesize rain and fog, and a triplet was constructed for subsequent work.

[0216] Specific implementation method nine: Figures 1 to 5 As shown, step 202 specifically includes:

[0217] Parallel attention module; contains a gated attention, a channel attention and pixel attention; batch normalizes the input features before performing parallel attention-based feature extraction;

[0218] Gated attention can initially extract pixel-level feature information. It consists of a feature extraction branch and a gating branch. The feature extraction branch includes convolutions with kernel sizes of 1×1 and 3×3, and the gating branch includes 1×1 convolution and Sigmoid activation function.

[0219] Channel attention can effectively extract global information across channels. By assigning different importance to the features of different channels, the model can focus more on features that are more useful for the current deraining task. It first uses a global average pooling operation to extract the global spatial information of each channel, then uses point-by-point convolution to learn the complex dependencies between channels, and uses the GELU activation function to increase nonlinearity. The Sigmoid activation function generates an attention score for each channel to ensure that the attention score is between 0 and 1. Finally, the attention score of each channel is multiplied by the corresponding channel of the original feature map to obtain a weighted feature map. This strengthens important feature channels and suppresses unimportant channels.

[0220] Pixel attention can extract global pixel-gated features. It focuses more on the importance of each pixel in the feature map, allowing the model to weight features at a finer level, thereby more accurately focusing on local features that are more important for the current deraining task. It first uses point-by-point convolution and GELU activation functions to learn from the input feature map and generate the corresponding pixel-level attention map. Then, the Sigmoid activation function is used to ensure that the attention score is between 0 and 1, and finally the original feature map is subjected to feature weighting operations.

[0221] The three different attention results are connected along the channel dimension, and the dimension of the connected feature channel is reduced to the same dimension as the input using point-by-point convolution-GELU-point-by-point convolution, and the output feature map is summed up using skip connection;

[0222] Optimal selection of sparse attention module; for a layer-normalized feature map First, point-by-point convolution with a kernel size of 1×1 is used to aggregate pixel-level cross-channel features, and depth-wise convolution with a kernel size of 3×3 is used to encode channel-level spatial information, thereby obtaining the query (Q), key (K), and value (V) mapped according to the feature map X;

[0223]

[0224] in represents the depthwise convolution, represents point-wise convolution;

[0225] The original self-attention mechanism calculates the similarity between each token and other tokens to obtain an attention map;

[0226] The optimal selection mechanism is introduced to perform feature selection. After obtaining the query (Q), key (K) and value (V), the similarity between all queries and keys is calculated by performing a dot product operation on Q and K to obtain the attention map. Only the largest k self-attention score values ​​are retained, and the other self-attention score values ​​are set to negative infinity; finally, the softmax function is used to normalize the attention scores to obtain the attention weights;

[0227] Multiply the attention weight by the value (V) to calculate the weighted sum of self-attention and get the self-attention output for each position, which is expressed as follows:

[0228]

[0229] Mixed-scale feedforward networks;

[0230] For an input feature map After layer normalization, 1×1 convolution is used to expand the number of channels, and then they are sent to two parallel multi-scale deep convolution paths, and 3×3 and 5×5 deep convolutions are used to enhance multi-scale local information extraction, respectively, which is expressed as:

[0231]

[0232] Among them, f 1×1 represents 1×1 convolution, f 3×3 and f 5×5 They represent 3×3 and 5×5 depth convolutions respectively, and concat(·) represents cross-channel connection;

[0233] Attention-enhanced fusion network;

[0234] For the features F from RGB images and infrared images RGB and F IR First, the global spatial information is integrated into the corresponding channel descriptor through an adaptive mean pooling operation; in order to thoroughly capture the dependencies between different channels, a linear layer is used for mapping, and then GELU activation is used. This process generates the RGB image feature weight W RGB and infrared image feature weight W IR , expressed as:

[0235]

[0236] Among them, f GAP represents global mean pooling, f MLP represents a multi-layer perceptron;

[0237] A correlation attention matrix A is used to weigh the features F from the RGB image and the infrared image. VIS and F IR the relationship between;

[0238]

[0239] Where μ(·) represents the Sigmoid activation function; the coefficient c represents the number of channels and acts as a proportional factor for attention enhancement; It means element-wise multiplication;

[0240] In the shared feature enhancement branch, we first combine the relevant attention matrix A with the RGB feature F RGB and infrared signature F IR Multiply by channel to get the initial fusion features;

[0241] The fused features are processed through the cross-channel fusion enhancement module:

[0242]

[0243] Among them, ⊙ represents channel-wise multiplication, concat(·) represents cross-channel concatenation, and f 7×7 Represents a depthwise convolution with a convolution kernel size of 7×7;

[0244] Simply perform complementary feature attention enhancement, recorded as:

[0245]

[0246] The spatial attention mechanism is used to aggregate the enhanced RGB and infrared features as output, and the fused features are obtained in the following way:

[0247]

[0248] Among them, f 1×1 Indicates that the convolution kernel size is 1×1 convolution;

[0249] Will get Split to obtain the fused RGB image features and infrared image features Add them together to get the fusion feature F fuse ;

[0250]

[0251] Among them, split(·) means splitting by channel;

[0252] Attention-guided expert networks;

[0253] The attention-guided expert network contains a total of 8 expert networks, namely: an average pooling layer with a receptive field of 3×3, a separable convolution layer with convolution kernel sizes of 1×1, 3×3, 5×5, and 7×7, and an expanded convolution layer with convolution kernels of 3×3, 5×5, and 7×7;

[0254] For the input feature map First, use the adaptive pooling layer to get the descriptor corresponding to each channel

[0255]

[0256] Among them, X(i, j, k) represents the pixel value at (i, j) of the kth channel of the feature map x;

[0257] The learnable weight matrices W1 and W2 are then used to obtain the coefficients T of each expert network:

[0258] T=W2σ(W1z),

[0259]

[0260] Where σ(·) represents the ReLU activation function, f 1×1 represents 1×1 convolution, represents the expert network operation, [·] represents the cross-channel connection;

[0261] The Charbonnier Loss loss function is used to train the IRDeNet network. The formula of the loss function is as follows:

[0262]

[0263] Among them, I gt Indicates a true rainless image, I derain Represents the rain-free image generated by IRDeNet.

[0264] Specific implementation method ten: Figures 1 to 5 As shown, step 203 specifically includes: respectively performing gt And the rain-free image after rain removal I Derain Use the pre-trained SAM model to segment the image, get their corresponding mask maps, and then construct a loss function to measure the gap between the two; the constructed loss function (seg loss) consists of focal loss and diceloss:

[0265]

[0266] Among them, α and β are adjustable hyperparameters;

[0267] Focal loss is used to solve the problem of imbalanced sample categories, which is defined as:

[0268]

[0269] Among them, p t It is expressed as:

[0270]

[0271] Dice loss is used to evaluate the similarity between two samples. It measures the performance of the segmentation model by comparing the similarity between the predicted results and the true labels. It is defined as:

[0272]

[0273] Among them, y i Indicates the value predicted by the model, ranging from 0 to 1, t i represents the target value;

[0274] The overall loss function of the IRDeNet network is expressed as:

[0275]

[0276] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for establishing a three-dimensional model for a simulation game, characterized in that: The specific steps include: Step 1, brightening the game scene conversion image with the three-dimensional model; Step 2: Remove rain from the 3D model game scene with rain; Step 3: Output the brightened and rain-removed 3D model game scene.

2. A method for establishing a three-dimensional model for a simulation game according to claim 1, characterized in that: Step 1 specifically includes: Step 101, obtaining a dataset image; Step 102: Dataset image enhancement; Step 103: training a brightness enhancement model based on the LIME method; Step 104: Model evaluation and optimization.

3. A method for establishing a three-dimensional model for a simulation game according to claim 2, characterized in that: Step 101 specifically includes: using automated web crawler technology to collect scene images whose details are difficult to identify; and storing all images in a data processing center after collection for subsequent image preprocessing.

4. A method for establishing a three-dimensional model for a simulation game according to claim 2, characterized in that: Step 102 specifically includes: for the collected scene image data with missing details, performing the following preprocessing operations to enhance the diversity of the data set and adapt to the model input requirements: Step 1021, rotation and mirroring: the image data is first randomly rotated and mirrored; the purpose of this operation is to increase the diversity of the data set by simulating different observation angles, thereby improving the robustness and generalization ability of the model in practical applications; Step 1022, random cropping: Randomly crop 256x256 pixel image blocks from the original coral image. This step ensures that all image blocks have a uniform size, which is suitable as the input of the deep learning model. This operation also helps the model learn to recognize and process images from local features, enhancing the model's ability to process various parts of the coral image.

5. A method for establishing a three-dimensional model for a simulation game according to claim 2, characterized in that: Step 103 specifically includes: the training of the brightness enhancement model is achieved by combining the LIME method with deep learning technology to ensure that the final model can effectively enhance the brightness and clarity of the scene image. The specific steps are as follows: Step 1031: Model architecture design; Design a deep learning model architecture based on a convolutional neural network. The model structure includes multiple convolutional layers, activation layers, and pooling layers. Each layer is designed to extract and process specific features of the image. The input layer of the model accepts a preprocessed scene image, and the output layer generates a brightness-enhanced scene image. Step 1032: data preparation and preprocessing; Before model training, necessary preprocessing is performed on the collected blurred scene images, using methods such as Gaussian blur and bilateral filtering to remove image noise while retaining important edge information; Step 1033, training the illumination map estimation network; A deep learning method based on Retinex theory is used. The training process is designed to enable the model to accurately learn and predict the illumination distribution under low-light conditions, thereby effectively enhancing the brightness of scene images. The specific formula of the established Retinex theory is as follows: I(x,y)=R(x,y)·L(x,y) Where I(x,y) represents the pixel value of the input scene image at position (x,y); R(x,y) represents the pixel value of the reflectivity map of the three-dimensional model in the scene at position (x,y); L(x,y) represents the pixel value of the three-dimensional model map in the scene at position (x,y); In order to train the model to accurately estimate illumination and reflectance, a joint loss function is used to minimize the error between the predicted value and the true value. This loss function includes two components: illumination estimation and reflectance estimation loss. By minimizing the error between the predicted value and the true value, the adaptability and accuracy of the model under different lighting conditions are improved. The joint loss function is defined as: Among them, α and β are weight parameters used to balance the importance of illumination loss and reflectivity loss; is the loss function of illumination estimation; is the loss function of reflectivity estimation; During the training process, the model uses the estimated light distribution to enhance the brightness of low-light coral images. By learning the characteristics of coral images under normal light conditions, the model accurately reconstructs the light map under low light conditions, improving the brightness and contrast of coral images that originally lack details; Step 1034: brightness adjustment and optimization; After the illumination map is estimated, histogram equalization and Gamma algorithm are applied to enhance the previously estimated illumination map to adjust the brightness and contrast of the image; Step 1035: image fusion and output; The enhanced scene image is fused with the original scene image with color distortion and missing details. Laplace pyramid fusion and band fusion-based image fusion technology are used to ensure that the two are seamlessly combined at the pixel level. This step accurately maintains the details of the original coral image while introducing the brightness advantage of the enhanced image through multi-scale decomposition and reconstruction. The fused image is significantly better than the original image in visual effect. It is not only brighter and clearer, but also maintains natural color expression.

6. A method for establishing a three-dimensional model for a simulation game according to claim 2, characterized in that: Step 104 specifically includes: The improved structural similarity index method is used to evaluate the scene image processed by the brightness enhancement network model. The brightness adjustment factor B and the contrast adjustment factor C are added to the SSIM formula. The improved SSIM calculation formula is: Among them, μ x , μ y Respectively represent the brightness mean of scene images X and Y, reflecting the average light intensity of the 3D model in the scene; σ x , σ y They represent the variance of scene images X and Y, respectively, representing the degree of illumination change in the scene; σ xy represents the covariance of scene images X and Y, describing the common trend of brightness changes of the two images; c1 and c2 are small constants used to ensure the stability of calculation and avoid the situation where the denominator is zero; B = |μ y ―μ x | represents the brightness adjustment factor, reflecting the change of average light intensity during the enhancement process; C = |μ y ―μ x | represents the contrast adjustment factor, which quantifies the change in image contrast during the enhancement process; For each pair of original low-light distorted and blurred scene images and visually perfect scene images in the database, the SSIM value is calculated using the processed detail-enhanced image and the perfect image to quantitatively evaluate the enhancement effect of the model; the SSIM value ranges from -1 to 1, where 1 indicates complete similarity; a high SSIM value indicates that the model effectively enhances the brightness and contrast of the image while maintaining the natural color and details of the image; finally, according to the SSIM evaluation results, parameter adjustment and algorithm iteration are performed; including adjusting network parameters such as learning rate, number of layers, and filter size to optimize model performance; in addition, the algorithm is iteratively optimized, including adjusting the calculation method of brightness and contrast adjustment factors to finely control the improvement of scene image quality.

7. A method for establishing a three-dimensional model for a simulation game according to claim 1, characterized in that: Step 2 specifically includes: Step 201, generating and screening rainy images; Step 202, RGB image deraining model; Step 203: Introduce global semantic guidance as additional supervision.

8. A method for establishing a three-dimensional model for a simulation game according to claim 7, characterized in that: Step 201 specifically includes: Step 2011, generating rain streaks through a variational inference framework; The principle of the rain streak synthesis network structure based on the variational inference framework is: (1) o=r+b. (1), Among them, o is a single rainy image, r is a rain pattern image, and b is a clean background image without rain; The rain pattern layer r is modeled as the following generator form: r=G θ (z).(2), Among them, z is a hidden variable, and θ represents the parameters of the generator; In order to ensure the isotropy of rainfall factors, the network assumes that the prior distribution of the latent variable z follows a normal distribution with a mean of 0 and a covariance of the identity matrix, that is; z~N(z|0,I).(3), Generate different rain patterns by adjusting the specific z dimension; Background modeling; Embed x into the Gaussian prior distribution of b; in, is a hyperparameter that measures the similarity between x and b; According to the above formula, the probability distribution of rain map o is: o~p θ (o|z,b). (5), Among them, p θ (·) indicates that the distribution also depends on the parameters θ of the generator; The task of generating rain images is transformed into the task of learning the general distribution p(o), which is expressed as follows: p(o)=∫∫p θ (o|z,b)p(z)p(b)dzdb. (6), Where p(z) and p(b) represent the prior distribution of z and b respectively; Hidden distribution learning based on variational inference framework; Using DIFFNet model to estimate monocular depth of RGB image I RGB After obtaining d(x), a hyperparameter β is used to adjust the required scattering intensity, and the transmission rate t(x) is obtained according to the following formula. t(x)=e -β×d(x) . (7), The hyperparameter A is used to adjust the color and intensity of the foggy image, and the foggy image I is obtained by the following formula: haze (x); I haze (x)=I RGB (x)×t(x)+A×(1-t(x)). (8), The screen blending model (SMB) is used to synthesize rain streaks and background images. The formula is as follows: R=B+SB⊙S. (9), Among them, R represents the synthesized rain image, B represents the rain-free background image, and S represents the rain pattern image; The dataset was cropped into 256×256 images and the VRGNet model was used to synthesize rain images with different directions, scales, and thicknesses. Finally, the above-mentioned ASM model was used to synthesize rain and fog, and a triplet was constructed for subsequent work.

9. A method for establishing a three-dimensional model for a simulation game according to claim 7, characterized in that: Step 202 specifically includes: Parallel attention module; contains a gated attention, a channel attention and pixel attention; batch normalizes the input features before performing parallel attention-based feature extraction; Gated attention can initially extract pixel-level feature information. It consists of a feature extraction branch and a gating branch. The feature extraction branch includes convolutions with kernel sizes of 1×1 and 3×3, and the gating branch includes 1×1 convolution and Sigmoid activation function. Channel attention can effectively extract global information across channels. By assigning different importance to the features of different channels, the model can focus more on features that are more useful for the current deraining task. It first uses a global average pooling operation to extract the global spatial information of each channel, then uses point-by-point convolution to learn the complex dependencies between channels, and uses the GELU activation function to increase nonlinearity; an attention score is generated for each channel through the Sigmoid activation function to ensure that the attention score is between 0 and 1; finally, the attention score of each channel is multiplied by the corresponding channel of the original feature map to obtain a weighted feature map. This strengthens important feature channels and suppresses unimportant channels; Pixel attention can extract global pixel-gated features. It focuses more on the importance of each pixel in the feature map, allowing the model to weight features at a finer level, thereby more accurately focusing on local features that are more important for the current deraining task. It first uses point-by-point convolution and GELU activation functions to learn from the input feature map and generate the corresponding pixel-level attention map. Then, the Sigmoid activation function is used to ensure that the attention score is between 0 and 1, and finally the original feature map is subjected to feature weighting operations. The three different attention results are connected along the channel dimension, and the dimension of the connected feature channel is reduced to the same dimension as the input using point-by-point convolution-GELU-point-by-point convolution, and the output feature map is summed up using skip connection; Optimal selection of sparse attention module; for a layer-normalized feature map First, point-by-point convolution with a kernel size of 1×1 is used to aggregate pixel-level cross-channel features, and depth-wise convolution with a kernel size of 3×3 is used to encode channel-level spatial information, thereby obtaining the query (Q), key (K), and value (V) mapped according to the feature map X; in represents the depthwise convolution, Represents point-wise convolution; The original self-attention mechanism calculates the similarity between each token and other tokens to obtain an attention map; The optimal selection mechanism is introduced to perform feature selection. After obtaining the query (Q), key (K) and value (V), the similarity between all queries and keys is calculated by performing a dot product operation on Q and K to obtain the attention map. Only the largest k self-attention score values ​​are retained, and the other self-attention score values ​​are set to negative infinity; finally, the softmax function is used to normalize the attention scores to obtain the attention weights; Multiply the attention weight by the value (V) to calculate the weighted sum of self-attention and get the self-attention output for each position, which is expressed as follows: Mixed-scale feedforward networks; For an input feature map After layer normalization, 1×1 convolution is used to expand the number of channels, and then they are sent to two parallel multi-scale deep convolution paths, and 3×3 and 5×5 deep convolutions are used to enhance multi-scale local information extraction, respectively, which is expressed as: Among them, f 1×1 represents 1×1 convolution, f 3×3 and f 5×5 They represent 3×3 and 5×5 depth convolutions respectively, and concat(·) represents cross-channel connection; Attention-enhanced fusion network; For the features F from RGB images and infrared images RGB and F IR First, the global spatial information is integrated into the corresponding channel descriptor through an adaptive mean pooling operation; in order to thoroughly capture the dependencies between different channels, a linear layer is used for mapping, and then GELU activation is used. This process generates the RGB image feature weight W RGB and infrared image feature weight W IR , expressed as: Among them, f GAP represents global mean pooling, f MLP represents a multi-layer perceptron; A correlation attention matrix A is used to weigh the features F from the RGB image and the infrared image. VIS and F IR the relationship between; Where σ(·) represents the Sigmoid activation function; the coefficient c represents the number of channels and acts as a proportional factor for attention enhancement; It means element-wise multiplication; In the shared feature enhancement branch, we first combine the relevant attention matrix A with the RGB feature F RGB and infrared signature F IR Multiply by channel to get the initial fusion features; The fused features are processed through the cross-channel fusion enhancement module: Among them, ⊙ represents channel-wise multiplication, concat(·) represents cross-channel concatenation, and f 7×7 Represents a depthwise convolution with a convolution kernel size of 7×7; Simply perform complementary feature attention enhancement, recorded as: The spatial attention mechanism is used to aggregate the enhanced RGB and infrared features as output, and the fused features are obtained in the following way: Among them, f 1×1 Indicates that the convolution kernel size is 1×1 convolution; Will get Split to obtain the fused RGB image features and infrared image features Add them together to get the fusion feature F fuse : Among them, split(·) means splitting by channel; Attention-guided expert networks; The attention-guided expert network contains a total of 8 expert networks, namely: an average pooling layer with a receptive field of 3×3, a separable convolution layer with convolution kernel sizes of 1×1, 3×3, 5×5, and 7×7, and an expanded convolution layer with convolution kernels of 3×3, 5×5, and 7×7; For the input feature map First, use the adaptive pooling layer to get the descriptor corresponding to each channel Where X(i, j, k) represents the pixel value at (i, j) of the kth channel of the feature map X; The learnable weight matrices W1 and W2 are then used to obtain the coefficients T of each expert network: T=W2σ(W1z), Where σ(·) represents the ReLU activation function, f 1×1 represents 1×1 convolution, represents the expert network operation, [·] represents the cross-channel connection; The Charbonnier Loss loss function is used to train the IRDeNet network. The formula of the loss function is as follows: Among them, I gt Indicates a true rainless image, I derain Represents the rain-free image generated by IRDeNet.

10. The method for establishing a three-dimensional model for a simulation game according to claim 7, characterized in that: Step 203 specifically includes: respectively performing the real rain-free image I gt And the rain-free image after rain removal I Derain Use the pre-trained SAM model to segment the image, get their corresponding mask maps, and then construct a loss function to measure the gap between the two. The constructed loss function (segloss) consists of two parts: focalloss and diceloss. Among them, α and β are adjustable hyperparameters; Focalloss is used to solve the problem of imbalanced sample categories, which is defined as: Among them, p t It is expressed as: Diceloss is used to evaluate the similarity between two samples. It measures the performance of the segmentation model by comparing the similarity between the predicted results and the true labels. It is defined as: Among them, y i Indicates the value predicted by the model, ranging from 0 to 1, t i represents the target value; The overall loss function of the IRDeNet network is expressed as:

Images