An artificial intelligence-based game sound effect data repairing method

Abstract

The application relates to the technical field of sound effect data repairing, in particular to a game sound effect data repairing method based on artificial intelligence. First, an original game sound effect data is expanded through a generative adversarial network integrated with a dynamic path selection mechanism to solve the sample singleness problem caused by the fixed structure of a traditional generative adversarial network; then, a self-encoder based on quantum state sparse constraint is adopted to carry out multi-level repairing on damaged sound effects, wherein adaptive noise suppression and a dynamic tuning feature feedback module are respectively adopted to process noise and spectrum distortion, so as to overcome the defects of a traditional self-encoder, such as insufficient reconstruction accuracy and poor detail capturing capability; finally, the outputs of multiple repairing stages are weighted and fused to generate high-fidelity and natural and coherent final sound effects, so that the problems of poor adaptability to spectrum characteristics and distorted sound quality after repairing in the prior art are effectively solved.

Description

Technical Field

[0001] This invention relates to the field of sound effect data restoration technology, and in particular to a method for restoring game sound effect data based on artificial intelligence. Background Technology

[0002] With the rapid development of the gaming industry, game sound effects play a vital role in enhancing player immersion and gaming experience. High-quality sound data not only increases the realism of the game but also helps to create the game atmosphere and improve the player's emotional experience. However, due to various factors in the sound acquisition process, game sound data is often affected by noise pollution, spectrum loss, or sampling distortion, leading to a decline in sound quality.

[0003] To address the aforementioned issues, existing technologies have proposed various audio processing solutions. For example, Chinese invention patent application CN202010670626.9 proposes an "audio signal processing method, apparatus, computer device, and storage medium," which uses a pre-trained spectrum compensation model to predict and compensate for the distortion spectrum in the input audio signal, and reconstructs the repaired target audio signal based on the prediction results, aiming to improve the versatility of speech signal repair. Another example is Chinese invention patent application CN202011296891.1, which proposes an "audio synthesis method, apparatus, device, and computer-readable storage medium," which acquires dry audio and corresponding phoneme data, performs phoneme error repair and cross-validation, and uses the cross-validated data to train a neural network model to improve the sound quality of the synthesized audio. Furthermore, generative adversarial networks and autoencoders are also frequently used for data repair and augmentation tasks.

[0004] However, these existing technologies have significant limitations. First, traditional generative adversarial networks (GANs) typically employ generators and discriminators with fixed structures. This fixed adversarial pattern easily leads to the generation of monotonous sound effect samples, lacking diversity and failing to cover the diverse sound effect needs of different game genres such as shooting, role-playing, and strategy games. Second, traditional autoencoder models suffer from insufficient reconstruction accuracy and poor detail capture when processing high-frequency, complex game sound effect data, making it difficult to achieve high-fidelity sound effect restoration and often resulting in sound quality distortion and decreased naturalness. Furthermore, existing methods generally lack adaptability to spectral characteristics and have limited effectiveness in correcting spectral holes or anomalous noise. Summary of the Invention

[0005] The purpose of this invention is to provide an artificial intelligence-based method for repairing game sound effect data, which solves the technical problems of traditional generative adversarial networks generating samples with limited variety, traditional autoencoders having insufficient reconstruction accuracy, and existing methods having poor adaptability to spectral characteristics.

[0006] To achieve the above objectives, this invention provides a method for repairing game sound effect data based on artificial intelligence, comprising the following steps:

[0007] We acquire and preprocess raw game sound effect data from various game engines and sound effect libraries to form an initial training set;

[0008] Using the initial training set, a generative adversarial network data augmentation model based on dynamic path selection is trained to generate augmented game sound effect data;

[0009] The original game sound effect data and the expanded game sound effect data are merged to form a reconstructed training set;

[0010] Using the reconstructed training set, train an autoencoder data reconstruction model based on quantum state sparse constraints;

[0011] The game sound effect data to be repaired is input into the trained data reconstruction model to obtain the first repaired game sound effect data, the second repaired game sound effect data and the third repaired game sound effect data.

[0012] The first, second, and third repaired game sound effect data are weighted and fused to obtain the final repaired game sound effect data.

[0013] Specifically, using the initial training set, a generative adversarial network data augmentation model based on dynamic path selection is trained to generate augmented game sound effect data, including:

[0014] The generative adversarial network data augmentation model based on dynamic path selection includes a decision network, a generator, and a discriminator.

[0015] A batch of game sound effect data is input into the decision network. The decision network dynamically calculates and outputs the generation path selection corresponding to each sample in the batch based on the time domain and frequency domain characteristics of the input data.

[0016] The generator selects the corresponding generation path and encodes the input noise vector to generate the corresponding game sound effect data through the corresponding generation path.

[0017] An adaptive spectrum filter network is used to perform spectrum compensation on the game sound effect data, and the filtered game sound effect data is output.

[0018] The discriminator receives the game sound effect data, the filtered game sound effect data, and the real game sound effect data, performs an authenticity judgment, and outputs the judgment result.

[0019] Based on the discrimination result, the loss function is calculated, and the parameters of the generator and the decision network are updated using the backpropagation algorithm;

[0020] Repeat the training until the preset training stopping condition is met to complete the model training.

[0021] Specifically, a batch of game sound effect data is input into the decision network. The decision network dynamically calculates and outputs the generation path selection corresponding to each sample in the batch based on the time-domain and frequency-domain characteristics of the input data. This includes:

[0022] The decision network is a three-layer fully connected neural network that processes the time-frequency domain feature vector of game sound effect data through a response function containing a Gaussian mixture model. The time-frequency domain feature vector includes the signal energy, zero-pass rate, and short-time energy in the time domain, as well as the spectral centroid, spectral bandwidth, and Mel-frequency cepstral coefficients in the frequency domain.

[0023] The generator selects a generation path and encodes the input noise vector to generate corresponding game sound effect data, represented as follows:

[0024]

[0025] In the formula, Represents a generator function. This represents the i-th noise vector. This indicates the j-th generated path selection. This represents the set of parameters for the generator. This represents the output function of the k-th generating module; This indicates the number of samples input into the generative adversarial network in the current batch, which is equivalent to the number of samples generated by the generator in the current batch, and also equivalent to the number of generation modules in the generator. The generation module is an internal module of the generator, and the structure of each module is the same as that of the generator in a traditional generative adversarial network. Its function is that each module receives different random noise vectors and generates different game sound effect data. It is the parameterized function of the k-th generation module.

[0026] Specifically, an adaptive spectrum filter network is used to perform spectrum compensation on the game sound effect data, and the filtered game sound effect data is output as follows:

[0027] ;

[0028] ;

[0029] ;

[0030] In the formula, This represents the i-th game sound effect data after processing by the adaptive spectrum filter. This represents the adaptive spectrum filter function. Indicates filter parameters; This represents the i-th game sound effect data initially generated by the generator. This represents the convolution operation. It is a filter kernel that is dynamically adjusted based on its input; and These are the weights and biases of the filter kernel, learned through game sound effect training data; This is the ReLU activation function.

[0031] Specifically, using the reconstructed training set, a data reconstruction model based on quantum state sparse constraints is trained, including:

[0032] The autoencoder data reconstruction model based on quantum state sparse constraints includes an encoding layer, a hidden layer, a decoding layer, an adaptive noise suppression module, and a dynamically tuned feature feedback module connected in sequence.

[0033] Specifically, training an autoencoder data reconstruction model based on quantum state sparse constraints using the reconstructed training set further includes:

[0034] The corrupted game sound effect data is input into the encoding layer, and the data is converted into a high-dimensional sparse representation through a dynamic sparse activation function;

[0035] The sparse representation, after being processed by the hidden layer, is input to the decoding layer to reconstruct the first repaired game sound effect data;

[0036] The first repaired game sound effect data is input into the adaptive noise suppression module for noise estimation and suppression, and the second repaired game sound effect data is output.

[0037] The second repaired game sound effect data is input into the dynamic tuning feature feedback module, its spectral characteristics are analyzed and the adjustment increment is calculated. The adjustment increment is added to the second repaired game sound effect data to obtain the third repaired game sound effect data.

[0038] Specifically, the process involves inputting damaged game sound effect data into the encoding layer, and then converting the data into a high-dimensional sparse representation using a dynamic sparse activation function.

[0039] The dynamic sparse activation function used in the coding layer is the Softplus function, denoted as: In the formula, It is a natural constant. For the dynamic sparse activation function itself, This is the input data for the autoencoder.

[0040] The present invention provides an artificial intelligence-based method for repairing game sound effect data. Through the high-precision repair of quantum state sparse constraint autoencoder and the compensation mechanism of adaptive spectrum filter, it can effectively repair the sound effect data distortion caused by sampling errors, loss or damage. The repaired sound effect data maintains high detail fidelity and the sound quality is clearer and more natural. It is particularly suitable for the repair needs of high-frequency and complex sound effects in game scenes.

[0041] The generative adversarial network method with dynamic path selection enables the generated sound effect samples to cover the needs of different game scenarios, improving sample diversity. This not only enriches the training dataset but also effectively enhances the model's generalization ability, reduces the model's dependence on few samples, and helps to address the specific needs of different game sound effects.

[0042] The adaptive spectrum filter and dynamic tuning feature feedback mechanism adjust parameters in real time to adapt to different spectrum characteristics, improving the sound quality and naturalness of the sound effect data. In particular, during the sound effect data expansion process, the adaptive spectrum filter automatically detects and corrects the phenomenon of spectral holes or excessive noise, thereby enhancing the spectral quality of the generated data and making it more in line with the requirements of game sound effects.

[0043] The weighted fusion repair strategy further integrates the features of the first, second, and third repaired sound effect data, optimizes the overall performance of the final repaired data, and enhances the naturalness, coherence, and applicability of the repaired sound effect data, enabling it to be smoothly applied to different game scenarios. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0045] Figure 1 This is a flowchart of the training process of the autoencoder algorithm based on quantum state sparse constraints of the present invention.

[0046] Figure 2 This is a flowchart of the steps of the game sound effect data repair method based on artificial intelligence of the present invention.

[0047] Figure 3 These are diagrams illustrating the effects of representing four different features in Embodiment 1 of the present invention.

[0048] Figure 4 This is a heatmap of the spectral repair quality for three states in Embodiment 2 of the present invention. Detailed Implementation

[0049] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0050] Please see Figures 1 to 4 ,in, Figure 1 This is a flowchart of the training process of the autoencoder algorithm based on quantum state sparse constraints of the present invention. Figure 2 This is a flowchart of the steps of the game sound effect data repair method based on artificial intelligence of the present invention.

[0051] This invention provides a method for repairing game sound effect data based on artificial intelligence, comprising the following steps:

[0052] S101: Acquire and preprocess raw game sound effect data from various game engines and sound effect libraries to form an initial training set;

[0053] Specifically, the training data comes from various game engines and sound effect libraries, covering different game types (such as shooting, role-playing, strategy, etc.) to provide comprehensive sound effect coverage. The training data is collected by using automated scripts to download data in batches from the APIs of game engines and sound effect libraries, while also utilizing contributed data from the game developer community and collecting it through open data interfaces. The collected data is stored in lossless compressed FLAC format, and all sound effect data files are stored hierarchically according to sound effect type and game type, with corresponding indexes created in the database. To adapt to the input requirements of the neural network structure of the artificial intelligence model, the collected training data is preprocessed, including missing value imputation, data normalization, and outlier smoothing operations, to form an initial training set.

[0054] S102: Using the initial training set, train a generative adversarial network data augmentation model based on dynamic path selection to generate augmented game sound effect data;

[0055] S103: Merge the original game sound effect data with the expanded game sound effect data to form a reconstructed training set;

[0056] Specifically, the acquisition, annotation, and preprocessing of training data are time-consuming and labor-intensive, and insufficient training samples can easily lead to poor model generalization ability and affect model accuracy. Therefore, this invention adopts a sample generation method based on generative adversarial networks (GANs) to expand game sound effect data. In traditional GANs, the generator and discriminator usually use the same network structure for adversarial learning, resulting in relatively homogeneous generated samples and difficulty in achieving effective sample diversity. The GAN of this invention dynamically adjusts through a preset decision network. The decision network analyzes the characteristics of the input data and guides the generator to select the most suitable generation path. The selection of the generation path is based not only on the characteristics of the data itself but also on the application scenario of the data, such as the different types of game sound effect requirements. In addition, the discriminator of this invention is designed to effectively distinguish the outputs generated by different generation paths, further improving the model's generalization ability and effect, and achieving rich diversity of generated samples.

[0057] Furthermore, the training process for the generative adversarial network algorithm used to generate game sound effect data is as follows:

[0058] 1. Select a batch of game sound effect training data and input it into the decision network. The decision network dynamically determines the most suitable generation path for this batch of game sound effect data based on the time domain and frequency domain characteristics of the game sound effect data. The decision network is a 3-layer fully connected neural network.

[0059] Set the game sound effects training dataset as The first one in the game sound effects training dataset The data is Each sample corresponds to one game sound effect data point, and the output of the decision network determines the generation path, represented as:

[0060]

[0061] In the formula, The parameters represent the decision network, specifically including the weights and biases of the decision network; This indicates the generation path selection corresponding to the i-th sample; For decision network functions.

[0062] Specifically, the decision network dynamically adjusts the most suitable generation path for this batch of data based on the time-domain and frequency-domain characteristics of the data. The implementation of the decision network function is expressed as follows:

[0063]

[0064] In the formula, It is the Sigmoid activation function; This represents the weight of the j-th neuron in the decision network; It is the response function of the j-th neuron to the input sample; The number of neurons in the decision network; It is the bias term of the decision network.

[0065] In one embodiment, the response function uses a Gaussian mixture model to process the time and frequency domain features of the game sound effect data, and is expressed as:

[0066]

[0067] In the formula, and These are the mean vector and variance parameter corresponding to the j-th neuron in the decision network, respectively. It is an L2 norm; This is a time-frequency domain feature calculation function used to calculate the time-frequency features of game sound effect data, including signal energy, zero-crossing rate, and short-time energy in the time domain, and spectral centroid, spectral bandwidth, and Mel-frequency cepstral coefficients in the frequency domain. The time-frequency features are concatenated to form a time-frequency feature vector, i.e., This is the time-frequency feature vector.

[0068] 2. Based on the output of the decision network, a specific generation path is activated. This generation path refers to the way the generator encodes noise vectors into feature vectors that approximate real game sound effect data. The generator produces game sound effect data through the selected path. This game sound effect data aims to mimic the real sound effects in the original game sound effect dataset. The way the generator produces game sound effect data is represented as follows:

[0069]

[0070] In the formula, The first generated by the generator Game sound effect data; For the input to the generator A noise vector is used to initiate the generation process; Represents a generator function; These are the parameters for the generator.

[0071] In one embodiment, the generator produces game sound effect data through a selected path as follows:

[0072]

[0073] In the formula, This represents the output function of the k-th generating module; This indicates the number of samples input into the generative adversarial network in the current batch, which is equivalent to the number of samples generated by the generator in the current batch, and also equivalent to the number of generation modules in the generator. The generation module is an internal module of the generator, and the structure of each module is the same as that of the generator in a traditional generative adversarial network. Its function is that each module receives different random noise vectors and generates different game sound effect data, thereby improving the richness of the data generated by the generative adversarial network. It is the parameterized function of the k-th generation module.

[0074] Furthermore, the implementation of the output function of the generation module is represented as follows:

[0075]

[0076] In the formula, It is the weight of the m-th input noise element in the k-th generation module. It is the path selection weight; For the input to the generator The m-th noise vector of each module; This represents the total amount of noise input into the generator's generation module.

[0077] Furthermore, the implementation of the parameterized function of the generation module is expressed as follows:

[0078]

[0079] In the formula, It is the adjustment factor of the k-th generating module; These are the parameters for the k-th generation module. Preferably, Set it to 0.1.

[0080] 3. During the generation process, to address sound quality damage or distortion, this invention utilizes an adaptive spectrum filter network as an auxiliary tool for the generator to simulate the spectral characteristics of the generated game sound effect data. It is particularly suitable for simulating spectral anomalies caused by insufficient or incorrect game sound effect data acquisition, such as spectral holes or excessive noise. Specifically, the adaptive spectrum filter network employs a multi-layer convolutional neural network, with each layer designed with an adaptive filtering mechanism. This mechanism dynamically adjusts the filtering parameters based on the spectral characteristics of the input game sound effect data. The output calculation method of the adaptive spectrum filter network is expressed as follows:

[0081]

[0082] In the formula, This represents the adaptive spectrum filter function. Indicates filter parameters.

[0083] Furthermore, the implementation of the filter function is expressed as follows:

[0084]

[0085] In the formula, This represents the convolution operation. It is a filter kernel that is dynamically adjusted based on its input.

[0086] In one embodiment, the parameterized form of the filter kernel is expressed as:

[0087]

[0088] In the formula, and These are the weights and biases of the filter kernel, learned through game sound effect training data; This is the ReLU activation function.

[0089] 4. The discriminator receives game sound effect data and real game sound effect data from the generator, evaluates whether the generated sound effects are sufficient to fool the discrimination mechanism, and feeds back the evaluation results to the generator to optimize the selection of the generation path and the generation strategy. The discrimination mechanism is the way the discriminator judges the authenticity of the input samples, represented as:

[0090]

[0091] In the formula, The discriminator's judgment result. These are the parameters for the discriminator; Here is the discriminant function of the discriminator; For the first Real game sound effect data.

[0092] In one embodiment, the discriminant function of the discriminator is implemented as follows:

[0093]

[0094] In the formula, It is the weight of the nth discriminant, where the discriminant is the number of neurons in the last layer of the discriminator; In this embodiment, the number of discriminants is 2, that is, one discriminant represents a real sample and the other discriminant represents a fake sample; For the Softmax function; It is the feature matching function of the nth discriminant; For the first The game sound effect data after adaptive filtering.

[0095] In one embodiment, Euclidean distance is used to measure the similarity between generated game sound effect data and real game sound effect data. The feature matching function is calculated as follows:

[0096]

[0097]

[0098] In the formula, It is the L2 norm, calculated in the same way as the Euclidean distance; It is a bias term.

[0099] 5. Based on the feedback from the discriminator, update the parameters of the generator and decision network using the backpropagation algorithm, and adjust them according to their performance to ensure that they can continue to effectively recognize the generated game sound effect data. The update formula is expressed as follows:

[0100]

[0101]

[0102] In the formula, This represents the updated generator parameters. This represents the updated discriminator parameters; This represents the loss function of the generative adversarial network, used to evaluate the difference between generated game sound effect data and real game sound effect data, as well as the accuracy of path selection; To generate the learning rate for the adversarial network. Preferably, Set to 0.01.

[0103] 6. Repeat the above steps until a preset stopping iteration condition is met, indicating that the model training is complete. In one embodiment, the preset stopping iteration condition is reaching a preset maximum number of iterations; preferably, the preset maximum number of iterations is set to 1000.

[0104] After the data augmentation model is trained, the number of samples is increased using the trained data augmentation model.

[0105] It should be noted that the purpose of the data augmentation model is not to repair the game sound effect data, but to simulate and augment more damaged game sound effect data so that the training samples for the data reconstruction model are sufficient.

[0106] In one embodiment, if the original collected game sound effect data consists of 800 records and the game sound effect data generated by the data augmentation model is 200 records, then the augmented dataset contains 1000 records of game sound effect data.

[0107] S104: Using the reconstructed training set, train an autoencoder data reconstruction model based on quantum state sparse constraints;

[0108] S105: Input the game sound effect data to be repaired into the trained data reconstruction model to obtain the first repaired game sound effect data, the second repaired game sound effect data and the third repaired game sound effect data;

[0109] Specifically, the expanded game sound effect data is used to train a data reconstruction model. The function of the data reconstruction model is to repair the game sound effect data by reconstructing the data. The data reconstruction model adopts an autoencoder based on quantum state sparsity constraints. On the basis of traditional autoencoders, it enhances the reconstruction capability of sound effect data by simulating the sparsity characteristics of quantum states, and performs high-precision repair of game sound effect data damaged during acquisition or transmission.

[0110] Furthermore, the training process of the autoencoder algorithm based on quantum state sparsity constraints is as follows:

[0111] 1. Initialize the parameters of the autoencoder algorithm, including the weights and biases. The initialization method is as follows:

[0112]

[0113]

[0114] In the formula, This means that the mean is 0. The variance follows a normal distribution. This indicates that it follows a specific distribution; This represents the initial variance of the autoencoder parameters; Represents a normal distribution; and These represent the initial weights and biases of the autoencoder, respectively. and These represent the weights and biases of the autoencoder, respectively.

[0115] 2. Input the damaged game sound effect data into the autoencoder. The damaged game sound effect data includes the original game sound effect data and the game sound effect data generated by the data augmentation model. The data is first encoded by the encoding layer of the autoencoder. The encoding layer of the autoencoder uses a quantum state sparse encoding strategy to convert the data into a high-dimensional sparse representation. The sparse representation is then processed by multiple hidden layers of the autoencoder using a nonlinear activation function, as follows:

[0116]

[0117] In the formula, This is the sparse representation after encoding; The input data is for the autoencoder, specifically the damaged game audio data; These are the weights of the coding layer in the autoencoder; The bias of the coding layer of the autoencoder; It is a dynamic sparse activation function.

[0118] In one embodiment, the dynamic sparse activation function is chosen as the Softplus function to simulate the nonlinear sparse characteristics of quantum states, making the activation of each coding unit closer to the probability distribution of the quantum state, thus possessing dynamic sparse activation capability based on quantum behavior. The implementation is expressed as follows:

[0119]

[0120] In the formula, It is a natural constant.

[0121] 3. The data processed by the hidden layer flows to the decoding layer of the autoencoder. The task of the decoding layer is to restore the high-dimensional sparse data to an approximation of the original sound effect data. The implementation of the decoding layer of the autoencoder is represented as follows:

[0122]

[0123] In the formula, and These are the weights and biases of the decoding layer of the autoencoder; It is a nonlinear transformation function; This is the output data of the autoencoder, specifically the game sound effect data for the first fix.

[0124] In one embodiment, the nonlinear transformation function enhances the model reconstruction stage's ability to capture details and dynamic range, and is implemented as follows:

[0125]

[0126] In the formula, It is a nonlinear transformation function.

[0127] 4. An adaptive noise suppression module is used in real-time during the reconstruction phase of the data reconstruction model training to analyze the reconstructed audio data and suppress potential noise. By analyzing the spectral characteristics of the reconstructed data, its noise suppression parameters are automatically adjusted to optimize the output sound quality and ensure the naturalness and applicability of the reconstructed sound effects. Specifically, the output of the decoder layer of the autoencoder is used as the input to the adaptive noise suppression module for noise analysis and suppression processing, as shown below:

[0128]

[0129] In the formula, This is game audio data with adaptive noise suppression, specifically the second-repaired game audio data; It is a noise estimation function based on the output of the decoding layer of the autoencoder; These are the parameters of the noise estimation function output by the autoencoder-based decoding layer.

[0130] In one embodiment, the noise estimation function output by the autoencoder-based decoding layer is implemented as follows:

[0131]

[0132] In the formula, and They are the first The weights, mean, and variance of each Gaussian component are automatically learned from the training data to best suppress noise. This represents the number of Gaussian components. Preferably, Set to 3, corresponding to the 3 dimensions.

[0133] 5. To address potential distortion or over-smoothing issues in specific frequency bands during game sound effect data reconstruction, which could lead to loss of detail, especially in the high-frequency range, a dynamic tuning feature feedback module is used to analyze the spectral characteristics of the reconstructed sound effects and adjust the parameters of relevant network layers accordingly. This ensures the model achieves optimal reconstruction quality across various sound effect scenarios. The calculation method is as follows:

[0134]

[0135] In the formula, It is an adjustment increment calculation function based on sound effect characteristics; The output is processed by the dynamic tuning feature feedback mechanism, specifically the third-repaired game sound effect data; The parameters of the adjustment increment calculation function are based on sound effect features.

[0136] Furthermore, the implementation method of the adjustment increment calculation function based on sound effect features is expressed as follows:

[0137]

[0138] In the formula, and These are the weights and biases of the dynamic tuning feature feedback module, respectively. It is a coefficient that controls the intensity of adjustment; It is the hyperbolic tangent function. Preferably, Set it to 0.2.

[0139] 6. Perform backpropagation of the error and update the parameters of the autoencoder. Calculate the gradient of the loss function based on the error, and use gradient descent to update the parameters of the autoencoder model, optimizing its reconstruction performance. Use the autoencoder's loss function to constrain the training process, as follows:

[0140]

[0141] In the formula, The loss function of the autoencoder. The magnitude of the game sound effect data reconstruction error is partially characterized by the calculation of the L2 norm. This is the function for calculating the regularization term of the autoencoder.

[0142] Furthermore, the regularization term in the autoencoder increases the sensitivity of the loss function and adjusts its optimization capability. Its weight-based Frobenius norm controls the model's complexity and prevents overfitting. The calculation method is expressed as follows:

[0143]

[0144] In the formula, This represents the square of the Frobenius norm of the weight matrix of the autoencoder; It is the Frobenius norm; It is the regularization coefficient of the autoencoder.

[0145] Furthermore, the parameters of the autoencoder are updated using gradient descent, and the update method is expressed as follows:

[0146]

[0147]

[0148] In the formula, This indicates a parameter update operation; is the learning rate of the autoencoder; This is the gradient of the loss function of the autoencoder with respect to its biases; This is the gradient of the loss function of the autoencoder with respect to its weights; It is an adjustment increment calculation function based on sound effect characteristics; The output is processed by the dynamic tuning feature feedback mechanism, specifically the second repaired game sound effect data; The parameters of the adjustment increment calculation function are based on sound effect features.

[0149] 7. Repeat the above steps until the preset stopping iteration condition is met, indicating that the model training is complete. In one embodiment, the preset stopping iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000.

[0150] S106: The first repaired game sound effect data, the second repaired game sound effect data, and the third repaired game sound effect data are weighted and fused to obtain the final repaired game sound effect data.

[0151] Specifically, after the autoencoder is trained, the output of the autoencoder (i.e., the first repaired game sound effect data) is used to perform feature fusion with the second repaired game sound effect data and the third repaired game sound effect data. The fused data is the final repaired game sound effect data.

[0152] In one embodiment, the fusion method is weighted average fusion, that is, feature fusion is performed on the first repaired game sound effect data, the second repaired game sound effect data and the third repaired game sound effect data, and the feature values ​​of the corresponding indices are weighted and averaged. The weights of the first repaired game sound effect data, the second repaired game sound effect data and the third repaired game sound effect data are 0.7, 0.2 and 0.1 respectively.

[0153] By using the high-precision repair mechanism of quantum state sparse constraint autoencoder and the compensation mechanism of adaptive spectrum filter, it is possible to effectively repair the audio data distortion caused by sampling errors, loss or damage. The repaired audio data maintains high detail fidelity and the sound quality is clearer and more natural. It is especially suitable for the repair needs of high-frequency and complex audio effects in game scenarios.

[0154] The generative adversarial network method with dynamic path selection enables the generated sound effect samples to cover the needs of different game scenarios, improving sample diversity. This not only enriches the training dataset but also effectively enhances the model's generalization ability, reduces the model's dependence on few samples, and helps to address the specific needs of different game sound effects.

[0155] The adaptive spectrum filter and dynamic tuning feature feedback mechanism adjust parameters in real time to adapt to different spectrum characteristics, improving the sound quality and naturalness of the sound effect data. In particular, during the sound effect data expansion process, the adaptive spectrum filter automatically detects and corrects the phenomenon of spectral holes or excessive noise, thereby enhancing the spectral quality of the generated data and making it more in line with the requirements of game sound effects.

[0156] The weighted fusion repair strategy further integrates the features of the first, second, and third repaired sound effect data, optimizes the overall performance of the final repaired data, and enhances the naturalness, coherence, and applicability of the repaired sound effect data, enabling it to be smoothly applied to different game scenarios.

[0157] Example 1:

[0158] In one embodiment, the effect of quantum state sparse constraints is verified to demonstrate its unique advantages in feature representation. Experiments compare four different feature representation methods: an autoencoder without sparsity constraints, an autoencoder based on L1 sparsity constraints, a conventional sparse autoencoder, and the quantum state sparse constraint autoencoder proposed in this invention. The autoencoder without sparsity constraints imposes no sparsity requirements; L1 sparsity constraints promote feature sparsity through L1 regularization terms; the conventional sparse autoencoder employs sparsity constraints based on Kullback-Leibler divergence; and this technique simulates quantum state characteristics to achieve dynamic sparse activation. In the experiments, the feature activation values ​​learned by each method are extracted. The horizontal axis represents the magnitude of the feature activation value, and the vertical axis represents the probability density, reflecting the frequency of different activation values. The combination of histograms and probability density curves visually demonstrates the differences in feature distribution among different methods. Experimental results show that the feature distribution without sparsity constraints is relatively dispersed, with many features having moderately large activation values ​​and lacking obvious sparsity characteristics. The feature distribution of L1 sparse constraints exhibits a typical Laplace distribution, with most feature activation values ​​close to zero, but the distribution curve smoothly declines. Conventional sparse autoencoders exhibit an exponential distribution, achieving good sparsity but with a relatively uniform distribution pattern. This technique's feature distribution displays a unique bimodal characteristic: approximately 90% of feature activation values ​​are very close to zero, while about 10% of features have large activation values, forming a distinct long-tailed distribution. This distribution pattern simulates the energy level distribution characteristics of a quantum system, ensuring both feature sparsity and preserving high activation values ​​for a few important features, thus helping to maintain sparsity without losing key information. The probability density curve further confirms the special nature of this technique's distribution, exhibiting the highest probability density in the near-zero region while maintaining a certain probability quality in the region of large activation values, which helps improve the model's ability to capture sound effect details and improve restoration accuracy.

[0159] Example 2:

[0160] In one embodiment, a comparative analysis of spectrum restoration quality is conducted to analyze the core advantages of this technology in restoring damaged game sound effect spectra. The experiment compares three states: the original damaged sound effect spectrum, the effect of conventional filtering restoration, and the effect of this technology. Conventional technology employs a fixed-band compensation method based on a finite impulse response filter, with the following configuration parameters: five preset fixed-band compensation bands, and smoothing using a Hanning window function. This experiment presents the spectral energy distribution in the form of a heatmap, with the horizontal axis representing time frames and the vertical axis representing the frequency range (0-22050 Hz), and the color intensity indicating the spectral energy strength. Experimental results show that the original damaged spectrum has significant energy holes around 13000 Hz and 18000 Hz. While the conventional method fills these holes, it results in an overly smoothed overall spectrum, completely losing high-frequency details. In contrast, the spectrum restored by this technology not only accurately fills the holes but also completely preserves the detailed features of the original spectrum, specifically: 1) the energy fluctuation pattern in the low-frequency region; 2) the narrowband energy peaks in the mid-to-high frequency region; and 3) the time-varying characteristics of the overall spectrum. Experimental results verify the precise reconstruction capability of the quantum state sparse constraint autoencoder for the spectral structure, as well as the specific compensation mechanism of the adaptive spectral filter for hole repair.

[0161] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. An artificial intelligence-based game sound effect data repairing method, characterized in that, The method comprises the following steps: acquiring and preprocessing original game sound effect data from multiple game engines and sound libraries to form an initial training set; training a dynamic path selection-based generative adversarial network data augmentation model using the initial training set to generate augmented game sound effect data; merging the original game sound effect data and the augmented game sound effect data to form a reconstructed training set; training a quantum state sparse constraint-based autoencoder data reconstruction model using the reconstructed training set; inputting the game sound effect data to be repaired into the trained data reconstruction model to obtain first, second and third repaired game sound effect data; weighting and fusing the first, second and third repaired game sound effect data to obtain final repaired game sound effect data; training a dynamic path selection-based generative adversarial network data augmentation model using the initial training set to generate augmented game sound effect data, specifically comprising: the dynamic path selection-based generative adversarial network data augmentation model comprises a decision network, a generator and a discriminator; inputting a batch of game sound effect data into the decision network, which dynamically calculates and outputs a generation path selection corresponding to each sample in the batch according to the time domain and frequency domain features of the input data; the generator encodes the input noise vector to generate corresponding game sound effect data through the corresponding generation path according to the generation path selection; using an adaptive spectral filter network to perform spectral compensation on the game sound effect data to output filtered game sound effect data; the discriminator receives the game sound effect data, the filtered game sound effect data and real game sound effect data, performs authenticity discrimination and outputs a discrimination result; according to the discrimination result, a loss function is calculated, and the parameters of the generator and the decision network are updated through a back propagation algorithm; repeating the training until a preset training stop condition is met to complete the model training; training a quantum state sparse constraint-based autoencoder data reconstruction model using the reconstructed training set, specifically comprising: the quantum state sparse constraint-based autoencoder data reconstruction model comprises an encoding layer, a hidden layer, a decoding layer, an adaptive noise suppression module and a dynamic tuning feature feedback module connected in sequence; inputting the damaged game sound effect data into the encoding layer to convert the data into a high-dimensional sparse representation through a dynamic sparse activation function; after the sparse representation is processed by the hidden layer, it is input into the decoding layer to reconstruct the first repaired game sound effect data; inputting the first repaired game sound effect data into the adaptive noise suppression module to perform noise estimation and suppression, and outputting the second repaired game sound effect data; inputting the second repaired game sound effect data into the dynamic tuning feature feedback module to analyze its spectral characteristics and calculate an adjustment increment, adding the adjustment increment to the second repaired game sound effect data to obtain the third repaired game sound effect data. 2.The AI-based game sound effect data repairing method of claim 1, wherein A batch of game sound effect data is input into the decision network, and the decision network dynamically calculates and outputs a generated path selection corresponding to each sample in the batch according to time domain and frequency domain features of the input data, specifically comprising: The decision network is a three-layer fully connected neural network, which processes a time-frequency domain feature vector of the game sound effect data through a response function containing a Gaussian mixture model, and the time-frequency domain feature vector includes signal energy, zero-crossing rate and short-time energy in the time domain, and spectral centroid, spectral bandwidth and mel-frequency cepstrum coefficient in the frequency domain. 3.The AI-based game sound effect data repairing method of claim 1, wherein The generator encodes the input noise vector to generate corresponding game sound effect data through the corresponding generation path according to the generation path selection, denoted as: In the formula, denotes a generator function, denotes the i th noise vector, denotes the j th generation path selection, denotes a parameter set of the generator, denotes an output function of the k th generation module; denotes the number of samples input to the generative adversarial network in the current batch, which is equal to the number of samples generated by the generator in the current batch, and also equal to the number of generation modules in the generator. The generation module is an internal module of the generator, and the structure of each module is the same as that of the generator of the conventional generative adversarial network. Each module receives different random noise vectors and generates different game sound effect data. is a parameterized function of the k th generation module. 4.The AI-based game sound effect data repairing method of claim 1, wherein The game sound effect data is compensated by using an adaptive spectral filter network, and filtered game sound effect data is output, represented as: ； ； ； In the formula, represents the i th game sound effect data processed by the adaptive spectrum filter, represents the adaptive spectrum filter function, represents the filter parameter; represents the i th game sound effect data initially generated by the generator, represents the convolution operation, is the filter kernel based on the input dynamic adjustment; and is the weight and bias of the filter kernel, which is learned by the game sound effect training data; is the ReLU activation function. 5.The AI-based game sound effect data repairing method of claim 1, wherein The damaged game sound effect data is input into the encoding layer, and the data is converted into a high-dimensional sparse representation through a dynamic sparse activation function, specifically comprising: The dynamic sparse activation function used by the encoding layer is a Softplus function, denoted as wherein is a natural constant, is the dynamic sparse activation function itself, is the input data of the autoencoder.

Images