Method and device for establishing box simulation model, and storage medium

Through the neural network model, the problem that the box simulation solution in the existing technology cannot accurately simulate nonlinear acoustics and complex dynamic behaviors is solved, and the adaptability and mimicry of multiple audio data is improved. .

CN119002259AInactive Publication Date: 2024-11-22ZHUHAI SINCO INTELLIGENT TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411040305.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing box simulation scheme cannot accurately simulate nonlinear acoustic problems and complex dynamic behaviors, and the simulated sound effects are not adaptable to different audio samples, so it is impossible to realize box simulation of multiple audio data.

Method used

Using neural network model, the sound of the physical box is trained through a variety of raw audio data, and the box simulation model is established. Convolutional neural network, bidirectional long-term network and attention mechanism layer are used to achieve nonlinear acoustic problems. learning and simulation of complex dynamic behaviors.

Benefits of technology

The box simulation model is improved to the realization and adaptability of the box sound to the physical box sound, and can more accurately simulate the box response of different audio data, enhancing the ability to adapt to multiple sound sources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119002259A_ABST
    Figure CN119002259A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of audio equipment, and discloses a box simulation model establishment method, a storage medium and a device, and the method comprises the following steps: placing a to-be-simulated entity box in a quiet closed space, and arranging sound receiving equipment near a loudspeaker of the entity box; inputting the various original audio data into the entity box body, and respectively recording the sound of each original audio data played by the entity box body by using the sound receiving equipment to obtain box body audio data corresponding to each original audio data; establishing a neural network model; and training the neural network model by taking the original audio data as input and the box audio data as output to obtain a box simulation model of the entity box. According to the method provided by the embodiment of the invention, the adaptability of the box body simulation model to box body simulation of various sound sources can be improved, nonlinear problems and complex dynamic behaviors in the simulation process can be processed, and the simulation degree of entity box body simulation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of audio equipment, and in particular to a method for establishing a cabinet simulation model, a storage medium and a device. Background Art

[0002] Cabinet simulation is a key technology in the field of audio equipment. It allows audio engineers and designers to record the sounds of different cabinets in a software environment and use impulse response (IR) calculation to achieve the simulated sound. In existing solutions, cabinet simulation mainly uses impulse response measurement, which measures the cabinet's response to a specific impulse signal, captures the cabinet's acoustic characteristics, and then simulates the cabinet's frequency response through digital signal processing technology to achieve cabinet simulation. Although existing cabinet simulation solutions can handle linear acoustic problems well, they cannot achieve accurate simulation for nonlinear problems or situations where complex dynamic behaviors of the coupling of sound and cabinet structure need to be considered. In addition, existing cabinet simulation solutions use fixed impulse signal impulses to obtain the coefficients of the finite impulse response (FIR). Single sample data will cause the simulated sound effect to lose adaptability to other types of audio samples, and it is impossible to achieve cabinet simulation for a variety of different types of audio data. Summary of the invention

[0003] In order to overcome the deficiencies of the prior art, the object of the present invention is to provide a method for establishing a cabinet simulation model, which can improve the more accurate simulation of the sound effects of the physical cabinet and improve the adaptability to a variety of different types of audio.

[0004] To solve the above problems, the technical solution adopted by the present invention is as follows: A method for establishing a box simulation model comprises the following steps:

[0005] Place the physical box to be simulated in a quiet, enclosed space, and set up a sound receiving device near the speaker of the physical box;

[0006] Inputting a plurality of original audio data into the physical box, and using a sound receiving device to respectively record the sound of the physical box playing each of the original audio data, to obtain box audio data corresponding to each of the original audio data;

[0007] Build a neural network model;

[0008] The neural network model is trained using the original audio data as input and the box audio data as output to obtain a box simulation model of the physical box.

[0009] Compared with the prior art, the beneficial effect of the present invention is that: by taking a variety of different types of original audio data as input and taking the box audio data of the sound emitted by the physical box playing the original audio data as output to train the neural network model, the model can have the adaptability to simulate the box for different types of sound sources. At the same time, the neural network model is used for training to obtain the box simulation model, and the deep learning ability of the neural network model can be used to learn the nonlinear acoustic problems and complex dynamic behaviors of the box, thereby achieving a more accurate simulation of the impulse response process of the box, and improving the realism of the box simulation model to the sound simulation of the physical box.

[0010] In the above-mentioned method for establishing a box simulation model, in the step of establishing a neural network model, the neural network model sequentially includes an input layer, a convolutional layer, an activation layer, a pooling layer, a bidirectional long-short term network layer and a fully connected layer.

[0011] In the above-mentioned method for establishing the box simulation model, an attention mechanism layer is provided between the bidirectional long-short term network layer and the fully connected layer.

[0012] The method for establishing the cabinet simulation model described above, wherein the steps of taking the original audio data as input and taking the cabinet audio data as output to train the neural network model, and obtaining the cabinet simulation model of the physical cabinet include:

[0013] Divide the original audio data and the corresponding box audio data into a training set and a test set;

[0014] Inputting the original audio data in the training set into the neural network model, and obtaining the predicted output of the neural network model;

[0015] Calculating the loss function of the neural network model according to the predicted output and the box audio data in the training set, and updating the parameters of the neural network model using a back propagation algorithm according to the loss function;

[0016] Repeat the above steps until the neural network model converges;

[0017] The test set is input into the converged neural network model to verify the neural network model. If the verification is passed, the converged neural network model is the box simulation model of the physical box.

[0018] A training device for a cabinet simulation model comprises: an audio output device for outputting a plurality of different original audio data to a simulated physical cabinet; a sound receiving device for recording the sound of the physical cabinet playing the original audio data, and obtaining cabinet audio data corresponding to each type of the original audio data; an acquisition unit for acquiring the original audio data and the cabinet audio data, and dividing the original audio data and the cabinet audio data into a training set and a test set; and a model training unit for establishing a neural network model, and training the neural network model with the original audio data as input and the cabinet audio data as output, to obtain a cabinet simulation model.

[0019] The above-mentioned training device for the box simulation model, the model training unit includes: a model training module, used to establish the neural network model, and input the original audio data in the training set into the neural network model to obtain the predicted data output by the neural network model; a loss function calculation module, used to calculate the loss function of the neural network model according to the predicted data and the box audio data in the training set; a parameter updating module, used to update the parameters of the neural network model according to the loss function using the back propagation algorithm; a verification module, used to verify the neural network model using the test set after the neural network model converges.

[0020] The training device of the above-mentioned box simulation model, the neural network model established by the model training module sequentially includes an input layer, a convolution layer, an activation layer, a pooling layer, a bidirectional long-short term network layer, an attention mechanism layer and a fully connected layer.

[0021] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for establishing a box simulation model is implemented.

[0022] An electronic device comprises: a memory for storing a computer program; and a processor for implementing the above-mentioned method for establishing a cabinet simulation model when executing the computer program stored in the memory.

[0023] An audio device comprises: a front-stage input unit, used for receiving an original audio signal input by a front-stage device, and converting the original audio signal into corresponding original audio data; a cabinet simulation unit, including a plurality of cabinet simulation models established by the above-mentioned cabinet simulation model establishment method, and used for processing the original audio data using the selected cabinet simulation model to obtain cabinet audio data of a physical cabinet to be simulated; and a rear-stage output unit, used for converting the cabinet audio data into corresponding cabinet audio signals, and outputting the cabinet audio signals to a rear-stage device.

[0024] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of a method for establishing a cabinet simulation model according to an embodiment of the present invention;

[0026] Figure 2 is a structural diagram of a neural network model according to an embodiment of the present invention;

[0027] Figure 3 It is a principle block diagram of a device for establishing a box simulation model according to an embodiment of the present invention;

[0028] Figure 4 The figure is a functional block diagram of an audio device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The embodiments of the present invention are described in detail below. Figure 1 , an embodiment of the present invention provides a method for establishing a box simulation model, comprising the following steps:

[0030] Place the physical box to be simulated in a quiet, enclosed space, and set up a sound receiving device near the speaker of the physical box;

[0031] Inputting a plurality of original audio data into the physical box, and using a sound receiving device to respectively record the sound of the physical box playing each of the original audio data, to obtain box audio data corresponding to each of the original audio data;

[0032] Build a neural network model;

[0033] The neural network model is trained using the original audio data as input and the box audio data as output to obtain a box simulation model of the physical box.

[0034] The method for establishing a cabinet simulation model of an embodiment of the present invention trains a neural network model by taking a variety of different original audio data as input and taking the cabinet audio data of the sound emitted by the physical cabinet playing these original audio data as output, and training the model through a variety of sound sources to improve the model's adaptability to different sound sources, so that a single model can simulate the cabinet for a variety of different sound sources. At the same time, the deep learning ability of the neural network model is used to learn the acoustic characteristics of the physical cabinet, so that the model can handle nonlinear acoustic problems and simulations of complex dynamic behaviors, thereby improving the model's realism of the sound effect simulation of the physical cabinet and improving the user's experience. It can be understood that the original audio data can include performance audio data of various instruments such as guitar, bass, and drums, and can also include human voice audio data. Each type of original audio data can include multiple segments of different audio data from different tones, different tracks, and different singers, so as to increase the number of training samples and improve the training effect of the neural network model.

[0035] It is understandable that the architecture of the neural network model can be selected according to actual needs. Figure 2 In this embodiment, in order to improve the training effect of the neural network model, the neural network model adopts a convolutional neural network model that introduces a bidirectional long-short-term network and an attention mechanism, that is, a neural network model of a CNN+Bi LSTM+Attention structure, which includes an input layer, a convolutional layer, an activation layer, a pooling layer, a bidirectional long-short-term network layer, an attention mechanism layer, and a fully connected layer in sequence, wherein the input layer, the convolutional layer, the activation layer, and the pooling layer constitute a convolutional neural network layer. The introduction of a bidirectional long-short-term network can realize the effective utilization of the forward and backward feature information based on the input by stacking two layers of long-short-term networks, thereby improving the accuracy of prediction. The attention mechanism can assign different weights to different parts of the input in order to better capture important information in the input data, thereby further improving the efficiency and accuracy of the model in processing sequence data. The specific training process of the neural network model includes the following steps:

[0036] S1: Divide multiple segments of original audio data of instruments such as guitar and bass or human voice singing and their corresponding box audio data into training set and test set;

[0037] S2: Input the original audio data in the training set into the convolutional neural network layer to extract the local features of the original audio data;

[0038] S3: Input the local features of the original audio data output by the convolutional neural network layer into the bidirectional long-term short-term network layer to model the global data and extract global information;

[0039] S4: The global information extracted by the bidirectional long-term short-term network layer is input into the attention mechanism layer. The attention mechanism layer can learn the relationship between the current output and input, so as to perform weighted summation on the features that need attention.

[0040] S5: Input the weighted summation result into the fully connected layer to obtain the predicted data;

[0041] S6: Calculate the loss function of the model according to the predicted data and the box audio data in the training set, and update the parameters of the model using the back propagation algorithm according to the loss function;

[0042] S7: Repeat the above steps S1 to S6 until the model converges.

[0043] After the model converges, the converged model is verified with the test set. If it passes the verification, the converged model is the final cabinet simulation model. If the similarity between the predicted data generated after processing the original audio data of the test set and the cabinet audio data in the test set is lower than the expected threshold, it is necessary to increase the number of samples or adjust the hyperparameters of the neural network model and then retrain.

[0044] It is understandable that in some embodiments, if the amount of sample data is large enough, a validation set can be divided from the original audio data and the box audio data, and the hyperparameters such as the number of neurons, number of network layers and learning rate of the neural network model can be debugged through the validation set to obtain better training effects and accelerate the convergence speed of the model.

[0045] Based on the same inventive concept, Figure 3 , an embodiment of the present invention also provides a training device for training a cabinet simulation model, including an audio output device, a sound receiving device, an acquisition unit and a model training unit. The audio output device is used to output a variety of different original audio data to the simulated physical cabinet, and the sound receiving device is used to record the sound emitted by the physical cabinet playing the original audio data to generate corresponding cabinet audio data. The acquisition unit is used to acquire the original audio data and the cabinet audio data, and divide the original audio data and the cabinet audio data into a training set and a test set. The model training unit is used to establish a neural network model, and train the neural network model with the original audio data as input and the cabinet audio data as output to obtain a cabinet simulation model. It can be understood that the audio output device can be an electronic musical instrument such as an electric guitar, bass or electronic keyboard, or it can be an audio device that stores audio data of a variety of different musical instruments or human voice performances.

[0046] It is understandable that, in some embodiments, the model training unit includes a model training module, a loss function calculation module, a parameter updating module and a verification module. The model training module is used to establish a neural network model, and input the original audio data in the training set into the neural network model to obtain the predicted data output by the neural network model. The loss function calculation module is used to calculate the loss function of the neural network model based on the predicted data and the box audio data in the training set. The parameter updating module is used to update the parameters of the neural network model using the back propagation algorithm according to the loss function. The verification module is used to verify the neural network model using the test set after the neural network model converges.

[0047] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method for establishing the cabinet simulation model is implemented.

[0048] In some possible embodiments, various aspects of the method for establishing a box simulation model provided by the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of the method for establishing a box simulation model according to various exemplary embodiments of the present application described above in this specification.

[0049] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned method for establishing a box simulation model.

[0050] In one possible design, the processor may include one or more processing units, and the processor may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor. In some embodiments, the processor and the memory may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0051] The processor may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the output method of the landing area disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0052] As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Rand om Access Memory, RAM), static random access memory (Static RandomAccess Memory, SRAM), programmable read-only memory (Programmable Read On ly Memory, PROM), read-only memory (Read On ly Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic storage, disk, optical disk, etc. The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0053] By programming the processor, the code corresponding to the method for establishing the box simulation model described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1 The steps of the method for establishing the cabinet simulation model of the embodiment shown are as follows: How to design and program a processor is a technique known to those skilled in the art and will not be described in detail here.

[0054] Based on the same inventive concept, an embodiment of the present invention also provides an audio device, including a front-stage input unit, a cabinet simulation unit and a rear-stage output unit. The front-stage input unit is used to connect with the front-stage device, receive the original audio signal input by the front-stage device, and convert the original audio signal into corresponding original audio data. The cabinet simulation unit includes a plurality of cabinet simulation models established by the above-mentioned cabinet simulation model establishment method, and each cabinet simulation model uses a different model or brand of the physical cabinet when training. The cabinet simulation unit is used to process the original audio data using the selected cabinet simulation model to generate cabinet audio data of the physical cabinet to be simulated. The rear-stage output unit is used to connect with the rear-stage device, and is used to convert the cabinet audio data into a corresponding cabinet audio signal, and output the cabinet audio signal to the rear-stage device. The audio device of the embodiment of the present invention can simulate the sound source of the electric guitar, bass or box head simulation input into an audio signal of the sound emitted by the selected physical cabinet with a high degree of realism through the neural network model in the cabinet simulation unit, and output it to headphones or other monitoring devices, so that the live performance effect can be simulated and previewed in occasions where it is not convenient to connect the sound, providing convenience for the performer to practice and create.

[0055] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0056] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0059] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A method for establishing a box simulation model, characterized in that: The steps include: Place the physical box to be simulated in a quiet, enclosed space, and set up a sound receiving device near the speaker of the physical box; Inputting a plurality of original audio data into the physical box, and using a sound receiving device to respectively record the sound of the physical box playing each of the original audio data, to obtain box audio data corresponding to each of the original audio data; Build a neural network model; The neural network model is trained using the original audio data as input and the box audio data as output to obtain a box simulation model of the physical box.

2. The method for establishing a box simulation model according to claim 1, characterized in that: In the step of establishing a neural network model, the neural network model sequentially includes an input layer, a convolutional layer, an activation layer, a pooling layer, a bidirectional long short-term network layer and a fully connected layer.

3. The method for establishing a box simulation model according to claim 2, characterized in that: An attention mechanism layer is arranged between the bidirectional long short-term network layer and the fully connected layer.

4. The method for establishing a box simulation model according to claim 1, characterized in that: The step uses the original audio data as input and the box audio data as output to train the neural network model, and the step of obtaining the box simulation model of the physical box includes: Divide the original audio data and the corresponding box audio data into a training set and a test set; Inputting the original audio data in the training set into the neural network model, and obtaining the predicted output of the neural network model; Calculating the loss function of the neural network model according to the predicted output and the box audio data in the training set, and updating the parameters of the neural network model using a back propagation algorithm according to the loss function; Repeat the above steps until the neural network model converges; The test set is input into the converged neural network model to verify the neural network model. If the verification is passed, the converged neural network model is the box simulation model of the physical box.

5. A training device for a box simulation model, characterized in that: include: An audio output device, used to output a variety of different original audio data to the simulated physical box; A sound receiving device, used for recording the sound of the physical box playing the original audio data, and obtaining the box audio data corresponding to each type of the original audio data; An acquisition unit, used to acquire the original audio data and the box audio data, and divide the original audio data and the box audio data into a training set and a test set; The model training unit is used to establish a neural network model, and train the neural network model with the original audio data as input and the box audio data as output to obtain a box simulation model.

6. The training device of the box simulation model according to claim 5, characterized in that: The model training unit comprises: A model training module, used to establish the neural network model, and input the original audio data in the training set into the neural network model to obtain the predicted data output by the neural network model; A loss function calculation module, used to calculate the loss function of the neural network model according to the predicted data and the box audio data in the training set; A parameter updating module, used to update the parameters of the neural network model using a back propagation algorithm according to the loss function; A verification module is used to verify the neural network model using the test set after the neural network model converges.

7. The training device of the box simulation model according to claim 5, characterized in that: The neural network model established by the model training module sequentially includes an input layer, a convolutional layer, an activation layer, a pooling layer, a bidirectional long short-term network layer, an attention mechanism layer and a fully connected layer.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for establishing a box simulation model according to any one of claims 1 to 4 is implemented.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; The processor is used to implement the method for establishing a box simulation model according to any one of claims 1 to 4 when executing the computer program stored in the memory.

10. An audio device, characterized in that: include: A front-stage input unit, used to receive an original audio signal input by a front-stage device and convert the original audio signal into corresponding original audio data; A cabinet simulation unit, comprising a plurality of cabinet simulation models established by the cabinet simulation model establishment method according to any one of claims 1 to 4, and used for processing the original audio data using the selected cabinet simulation model to obtain cabinet audio data of the physical cabinet to be simulated; The post-stage output unit is used to convert the cabinet audio data into a corresponding cabinet audio signal and output the cabinet audio signal to a post-stage device.

Citation Information

Patent Citations

  • Lithium ion battery health state prediction method based on CNN-BiLSTM-AT hybrid model

    CN114325450A

  • Model training and speech synthesis method and device, equipment and medium

    CN115700871A

  • Speech recognition method, system, device and storage medium

    CN116259305A

  • Training method, recognition method and device of auditory brainstem response recognition model

    CN117828347A

  • Arm skeleton data classification prediction method based on visual perception

    CN118015363A