Audio frequency integration simulation prototype verification test platform

Abstract

The invention provides an audio frequency integration simulation prototype verification test platform comprising a user demand input module used for receiving user demand information comprising integrated system information and audio frequency to be verified; the simulation prototype building module is used for building a simulation prototype for processing the audio frequency to be verified according to the integrated system information; the scene tracking and scheduling module is used for tracking and recording the input and output of the audio frequency to be verified in each audio frequency processing module in the simulation prototype in real time by utilizing a timestamp and an event driving mechanism; and the verification module is used for verifying the processing effect of the to-be-verified audio frequency in each audio frequency processing module according to the input and output of the to-be-verified audio frequency in each audio frequency processing module in the simulation prototype. By implementing the method, the integration problem can be accurately positioned, the limitation of a traditional isolated test is broken through, the problem that the traditional test cannot simulate a complex scene is effectively solved, the feasibility and effectiveness of the technology are comprehensively verified, and the research and development cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of simulation test, and particularly relates to an audio integrated simulation prototype verification test platform. Background Art

[0002] In the current era of rapid digital and intelligent development, audio technology has been widely applied in many fields. With the increasing complexity and diversification of audio application scenarios, the requirements for audio integrated architectures are also getting higher and higher. An audio integrated architecture needs to integrate multiple links such as audio acquisition, processing, transmission, storage, and output to achieve high-quality and high-performance audio functions.

[0003] However, in the process of researching and developing audio integrated architectures, many challenges are faced. The feasibility and effectiveness of various key technologies, such as advanced audio processing algorithms, efficient scheduling strategies, and stable message communication mechanisms, need to be strictly verified. Traditional testing methods often can only conduct isolated tests on individual technical points, unable to simulate actual complex application scenarios, and difficult to discover compatibility problems, performance bottlenecks, and potential risks that may occur during the technology integration process. This results in problems such as poor audio quality and unstable systems in the actual application of products, increasing the R & D cost and time. Therefore, there is an urgent need to build an audio integrated simulation prototype verification test platform to meet the needs of audio integrated architecture technology research and development and verification. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an audio integrated simulation prototype verification test platform to meet the needs of audio integrated architecture technology research and development and verification.

[0005] To achieve the above purpose, the present invention provides the following technical solutions: The present invention provides an audio integrated simulation prototype verification test platform, including: a user requirement input module for receiving user requirement information, including integrated system information and the audio to be verified; a simulation prototype building module for building a simulation prototype for processing the audio to be verified according to the integrated system information; a scenario tracking and scheduling module for using a timestamp and an event-driven mechanism to track and record in real time the input and output of the audio to be verified in each audio processing module in the simulation prototype; a verification module for verifying the processing effect of the audio to be verified in each audio processing module according to the input and output of the audio to be verified in each audio processing module in the simulation prototype.

[0006] Optionally, if the integrated system information is integrated system index information, the simulation prototype building module includes: an integrated system configuration search module for retrieving the corresponding integrated system configuration list according to the integrated system index information; a first simulation prototype building sub-module for building a simulation prototype for processing the audio to be verified according to the integrated system configuration list.

[0007] Optionally, the integrated system configuration list includes: the serial number of the audio function module, the order of the audio function modules, the audio function parameters, the type of message middleware, and the hardware device parameters. Each audio function module incorporates an audio processing algorithm, and the audio processing algorithm includes a noise reduction algorithm, an equalizer algorithm, an audio encoding algorithm, a mixing algorithm, and an audio decoding algorithm.

[0008] Optionally, if the integrated system information is a document for constructing an audio integrated system to be verified, the simulation prototype building module includes: a configuration factor reading module for reading the document for constructing the audio integrated system to be verified and performing keyword recognition on the document for constructing the audio integrated system to obtain the integrated system configuration factors; a configuration factor matching module for performing fuzzy matching between the integrated system configuration factors and the names of the sub-modules that make up the integrated system pre-stored in the platform; a configuration parameter determining module for extracting the corresponding parameters of the target integrated system configuration factors from the document for constructing the audio integrated system to be verified, where the target integrated system configuration factors are the factors that match successfully with the names of the integrated system configuration list pre-stored in the platform; and a second simulation prototype building sub-module for building a simulation prototype for processing the audio to be verified based on the target integrated system configuration factors and their corresponding parameters.

[0009] Optionally, when the integrated system information includes environmental requirement information, the verification module includes: a virtual sound field environment generator determination module for extracting the environmental requirement information from the integrated system index information and determining the corresponding virtual sound field environment generator according to the environmental requirement information; a virtual sound field determination module for inputting the output audio of the simulation prototype into the corresponding virtual sound field environment generator to obtain a simulated audio; and a virtual sound field performance determination module for collecting the simulated audio and extracting the performance parameters from the simulated audio.

[0010] Optionally, the virtual sound field environment generator includes a generator and a discriminator, and the construction process of the virtual sound field environment generator includes: obtaining multiple training samples, each training sample contains random noise, real sound frequency in the target sound field environment, and environmental parameters in the target sound field environment; inputting multiple training samples into the virtual sound field environment generator, wherein the random noise and environmental parameters in the target sound field environment are input into the generator, the generator generates simulated sound frequency, the real sound frequency in the target sound field environment and the simulated sound frequency are input into the discriminator, and the corresponding probability value is output, which represents the probability that the input sound frequency is the real sound frequency; the probability output when the discriminator inputs the real sound frequency and the probability output when the simulated sound frequency is input are input into the discrimination loss function to obtain a discrimination loss value; the probability output when the simulated sound frequency is input is input into the generation loss function to obtain a generation loss value; according to the discrimination loss value and the generation loss value, the partial derivative of each parameter in the virtual sound field environment generator is calculated layer by layer, and the parameters are updated until the preset requirements are met to obtain a trained virtual sound field environment generator.

[0011] Optionally, the verification module includes: a user expectation input module for receiving the user's expectation of the audio signal output by the simulation prototype; an expected effect calculation module for treating the expected effect parameter adjustment of each audio processing module as the action of the intelligent agent based on reinforcement learning, and measuring the degree of proximity between the audio signal output by the simulation prototype and the user's expectation according to a preset reward function, so as to obtain the optimal expected effect of each audio processing module; a processing effect verification module for calculating the difference between the output of each audio processing module of the audio to be verified in the simulation prototype and the expected effect of the corresponding audio processing module, and using the difference calculation result as the processing effect verification result of the audio processing module; and a display module for displaying the processing effect verification result of each audio processing module.

[0012] Optionally, an audio integrated simulation prototype verification test platform further includes: an integrated system configuration list modification module, which is used to receive a user's instruction to modify the integrated system configuration list.

[0013] An embodiment of the present invention provides an audio integration simulation prototype verification test platform. By establishing a simulation prototype, a scenario tracking and scheduling module tracks the input and output of each audio module and performs verification, thereby accurately locating integration problems, breaking through the limitations of traditional isolated testing, and effectively solving the problem that traditional testing cannot simulate complex scenarios. It comprehensively verifies the feasibility and effectiveness of the technology and reduces R&D costs.

[0014] Other advantages, objectives and features of the present invention will be described in the following description and will be apparent to those skilled in the art to some extent, or those skilled in the art can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. Brief Description of the Drawings

[0015] To make the objectives, technical solutions and advantageous effects of the present invention clearer, the following drawings are provided for description of the present invention: Figure 1 It is a specific module example diagram of an audio integrated simulation prototype verification test platform provided by the present invention; Figure 2 It is a specific module schematic diagram of the simulation prototype building module in the present invention. Detailed Description of the Preferred Embodiments

[0016] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0017] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may also be the communication inside two components. It may be a wireless connection or a wired connection. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0018] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0019] An embodiment of the present invention provides an audio integrated simulation prototype verification test platform, as Figure 1 shown, including: A user requirement input module 101, configured to receive user requirement information, including integrated system information and an audio to be verified; A simulation prototype building module 102, configured to build a simulation prototype for processing the audio to be verified according to the integrated system information; A scenario tracking and scheduling module 103, configured to use a timestamp and an event-driven mechanism to track and record in real time the inputs and outputs of the audio to be verified in each audio processing module in the simulation prototype; A verification module 104, configured to verify the processing effects of the audio to be verified in each audio processing module according to the inputs and outputs of the audio to be verified in each audio processing module in the simulation prototype.

[0020] Exemplarily, the user requirement input module 101 can be a graphical user interface, which can be a touch screen interface. The graphical user interface displays various options and receives the user's touch option results. The graphical user interface can also include a text-based input interface and / or a file input interface. The integrated system information can be an integrated system index, which can be used to index the pre-stored audio integrated system parameters to be verified in the test platform. For example, the integrated system information is a specific number or a specific character, which corresponds to the pre-stored audio integrated system to be verified. The audio integrated system is stored in the form of data processing modules and parameters. That is, the entire audio integrated system can be completely restored through these modules and parameters. The integrated system information can also be text information, which records the necessary factors for building the integrated system, such as data processing modules and corresponding parameters. The integrated system information can also be the user's selection information and input parameters on the graphical user interface. Different data processing modules are displayed on the graphical user interface. The user can select the corresponding data processing module according to their own needs, and can change the parameters in the data processing module. In addition, the connection relationship and connection order between different data processing modules can also be selected.

[0021] The simulation prototype building module 102, when the user inputs very detailed integrated system information on the platform. For example, when the user inputs a detailed description in the text input interface of the graphical user interface, then this type of information is comprehensively analyzed and decomposed into different categories and levels. For example, distinguish hardware-related information and software-related information. The hardware-related information such as the model and performance parameters of the audio acquisition device; the software-related information such as the audio processing algorithm and software architecture used; and the overall system function requirements and performance indicators, etc. Key parameters are extracted from the parsed information, and these parameters will directly affect the building of the simulation prototype. For example, according to the specific requirements of the audio processing algorithm, key audio processing parameters such as the sampling rate, quantization bit number, and filter parameters are determined.

[0022] Then, based on the functional requirements in the system information, search in the predefined module library, which contains various developed audio processing modules, such as audio filtering modules, noise reduction modules, mixing modules, encoding and decoding modules, etc. For the selected modules, configure them according to the specific parameter requirements in the system information. Taking the filtering module as an example, it is necessary to determine parameters such as the type of filter (such as low-pass, high-pass, band-pass, etc.), cut-off frequency, and order according to the key parameters to make it meet the processing requirements of the simulation prototype. Finally, according to the audio processing flow and architecture requirements of the system, combine and connect the configured modules to form a complete audio processing link. For example, first connect the output of the audio acquisition module to the input of the noise reduction module, and then connect the output of the noise reduction module to the input of the mixing module, etc., to ensure that the audio can be transmitted and processed between the modules in the predetermined order and manner.

[0023] When the simulation prototype starts, the scene tracking and scheduling module 103 sets a global time reference for the entire system, such as the system clock, and creates an initial timestamp representing the start time of the simulation. At the same time, event listeners are set at the input and output ports of each audio processing module to be responsible for detecting the flow of audio data. When audio data arrives at the input port or is output from the output port, the corresponding listener will be triggered. Specifically, when the listener detects the occurrence of the corresponding event, it will trigger an event object. This event object contains various information related to the event, such as the event type, occurrence time, and the audio data involved. For example, when audio data is output from a mixing module, the triggered event object will record that this is a "mixing module output" event, as well as the current timestamp and the content of the output audio data.

[0024] The verification module 104, after obtaining the input and output data of the audio to be verified in each audio processing module from the scenario tracking and scheduling module, extracts various information of the audio from the input and output data, such as waveform, frequency, amplitude, etc. Further, the collected data can be preprocessed, including data cleaning to remove possible noise or incorrect data, data normalization to map the data to a specific range, and data format conversion to convert the data into a format suitable for processing by the verification algorithm. Then, the specific functions of each audio processing module are verified. For example, for an audio compression module, check whether its compression ratio reaches the expected value, whether the quality of the decompressed audio is within an acceptable range, and whether there are obvious distortions or information losses. Finally, according to the preset evaluation metrics corresponding to each module, calculate the metric values of the input and output data of the audio to be verified in each module, and compare them with the preset standard values. In addition, it is also possible to verify whether the processing flow of the audio in the entire simulation prototype is correct and reasonable as a whole. Check whether the connections between the modules are correct, whether the data transfer between the modules is smooth, and whether there is a decrease in audio quality caused by problems in the cooperation between the modules. Finally, conduct an in-depth analysis of the results obtained during the verification process to identify the modules or links where the processing effect does not meet the expectations. Analyze the possible causes of the problems, such as unreasonable module parameter settings, abnormal input data, or defects in the algorithm itself.

[0025] An embodiment of the present invention provides an audio integrated simulation prototype verification test platform. By establishing a simulation prototype, the scenario tracking and scheduling module tracks the input and output of each audio module and conducts verification, which can accurately locate integration problems, break through the limitations of traditional isolated testing, effectively solve the problem that traditional testing cannot simulate complex scenarios, comprehensively verify the feasibility and effectiveness of the technology, and reduce the R & D cost.

[0026] As an alternative implementation, when the integrated system information is the integrated system index information, the simulation prototype building module includes: An integrated system configuration search module for retrieving the corresponding integrated system configuration list according to the integrated system index information; The first simulation prototype building sub-module for building a simulation prototype for processing the audio to be verified according to the integrated system configuration list.

[0027] Exemplarily, in addition to the above-mentioned integrated system information being the detailed description input by the user in the text input interface of the graphical user interface, it can also be the integrated system index. When the integrated system information is the integrated system index, this embodiment gives the corresponding implementation.

[0028] Receive the integrated system index information input by the user, which can be a code. After receiving the integrated system index information, parse the information and extract the key identification information from the parsed index information, such as system type, version number, application scenario, etc. These key information will be used as the basis for subsequent searching the configuration list. For example, if the input index information contains "audio processing system 2.0 - concert hall scenario", then extract "audio processing system", "2.0", and "concert hall scenario" as the key information.

[0029] Store and connect the key information with the integrated system configuration list in the database. When the key information is received, perform a database query and feedback the filtered integrated system configuration list to the first simulation prototype building sub - model. The first simulation prototype building sub - model constructs a simulation model according to the integrated system configuration list. Specifically, the integrated system configuration list includes: audio function module serial number, audio function module order, audio function parameters, message middleware type, and hardware device parameters. Among them, each audio function module has an audio processing algorithm built - in. The audio processing algorithms include noise reduction algorithm, equalizer algorithm, audio encoding algorithm, mixing algorithm, audio decoding algorithm, etc.

[0030] The audio function module serial number is used to index the audio processing module corresponding to the function. The audio function module order is used to specify the connection order of the audio processing modules with different functions. The audio function parameters correspond to the audio processing module, indicating the parameters in each audio processing module. For example, if the audio processing module is an equalizer module and the audio function parameters specify the gain values of each frequency band, then these gain values will be passed to the constructor of the module when instantiating the equalizer module. The message middleware type is used to configure the corresponding message middleware. The message middleware is used for communication and data transfer between modules, such as Kafka, RabbitMQ, etc. The configuration content includes server address, port number, topic, etc.

[0031] After the first simulation prototype building sub - module obtains the integrated system configuration list, according to this list, call the corresponding audio processing module and determine the input - output connection relationship between each audio function module. For example, if the configuration list specifies the module order as "noise reduction module - equalizer module - audio encoding module", then connect the output of the noise reduction module to the input of the equalizer module, and connect the output of the equalizer module to the input of the audio encoding module. Then, adjust the corresponding parameters of each audio processing module according to the audio function parameters and hardware device parameters. Finally, configure the message middleware according to the message middleware type to achieve communication between each module. Further, after building the simulation model, the simulation model can be displayed in the form of a graph structure.

[0032] An embodiment of the present invention provides an audio integrated simulation prototype verification test platform, which opens an integrated system index information input interface for users. Users only need to input the corresponding index information to complete the verification of the stored integrated system, reducing the learning cost of users and minimizing user operations.

[0033] As an alternative implementation, as Figure 2 shown, if the integrated system information is a document for constructing an audio integrated system to be verified, the simulation prototype building module includes: A configuration factor reading module 201, configured to read the document for constructing the audio integrated system to be verified, and perform keyword recognition on the document for constructing the audio integrated system to obtain integrated system configuration factors; A configuration factor matching module 202, configured to perform fuzzy matching between the integrated system configuration factors and the names of the sub-modules that make up the integrated system pre-stored in the platform; A configuration parameter determination module 203, configured to extract the corresponding parameters of the target integrated system configuration factors from the document for constructing the audio integrated system to be verified, where the target integrated system configuration factors are the factors that successfully match the names of the integrated system configuration lists pre-stored in the platform; A second simulation prototype building sub-module 204, configured to build a simulation prototype for processing the audio to be verified according to the target integrated system configuration factors and their corresponding parameters.

[0034] Exemplarily, generally before an audio integrated system needs to be constructed, the design of the system will be completed in advance. For example, for development files, when a system construction document similar to such development files is input in the user requirement input module, this embodiment can provide the following modules to complete the reading, mining, and matching of the system construction document to implement the construction of a simulation model according to the document.

[0035] When receiving the document for constructing the audio integrated system, the natural language processing (NLP) technology is used to perform core content recognition on the read document content, and representative and key configuration factors are extracted as keywords. Specifically, a named entity recognition (NER) model based on deep learning, such as BERT, BiLSTM-CRF, etc., can be used to train the document content, so as to learn the context information and semantic features in the text, and accurately identify the named entities related to the audio integrated system configuration in the document, such as audio function module names, parameter names, message middleware types, and hardware device names, etc., as key configuration factors.

[0036] In this embodiment, a thesaurus containing keywords related to the audio integration system is predefined. These keywords include audio processing module names (such as filters, amplifiers, etc.) and parameter names (such as sampling rate, gain, etc.). For the extracted key configuration factors, they are cleaned according to the keyword thesaurus. Specifically, appropriate fuzzy matching algorithms are selected, such as the edit distance algorithm (such as Levenshtein distance), cosine similarity algorithm, etc., to measure the similarity between two strings. Then, the integrated system configuration factors are compared one by one with the names of the sub-modules that make up the integrated system pre-stored in the platform. For each configuration factor, the similarity score between it and each sub-module name is calculated. According to the preset similarity threshold, the matching results with similarity scores higher than the threshold are screened out, and these successfully matched configuration factors are marked as factors that match the sub-module names in the platform. Finally, after the matching is completed, the configuration parameter determination module can capture the parameter value after the parameter name in a regularized manner. The simulation process of the second simulation prototype building sub-module is as described in the corresponding above embodiment and will not be elaborated here.

[0037] The embodiment of the present invention provides an audio integration simulation prototype verification test platform. By means of first recognition and then matching, on the one hand, in the recognition process, natural language processing technology is used to recognize the core content of the document and extract configuration factors as keywords, which can deeply understand the semantics of the document, accurately find out representative and key configuration factors, rather than simply based on literal matching, avoiding the situation of missing important information or misidentifying non-key information as keywords. On the other hand, different audio integration system construction documents may have differences in expression methods, word usage habits, etc. Directly matching and recognizing keywords in the document may not achieve good results due to different document styles. However, the method of first extracting keywords and then performing fuzzy matching can normalize various expressions through natural language processing technology, extract configuration factors with different expressions but the same meaning and perform reasonable matching, which has stronger adaptability and flexibility.

[0038] As an optional implementation manner, when the integrated system information includes environmental requirement information, the verification module includes: A virtual sound field environment generator determination module, configured to extract environmental requirement information from the integrated system index information and determine the corresponding virtual sound field environment generator according to the environmental requirement information; A virtual sound field determination module, configured to input the output audio of the simulation prototype into the corresponding virtual sound field environment generator to obtain simulated audio; A virtual sound field performance determination module, configured to collect the simulated audio and extract performance parameters from the simulated audio.

[0039] Exemplarily, the environmental requirement information is used to represent the environmental information where the integrated system actually needs to be placed. For example, when the environment where the integrated system is actually used is a music classroom, the environmental information includes the size and shape of the music classroom, the sound absorption coefficient and reflection coefficient corresponding to the surrounding environmental materials, and the location where the integrated system is placed in the room.

[0040] The manner in which the virtual sound field environment generator determination module determines the corresponding virtual sound field environment generator according to the environmental requirement information can be to match the environmental requirement information with the information tags of the pre-trained virtual sound field environment generators, and the information tags contain the environmental information corresponding to the virtual generation environment generators. The virtual sound field environment generator includes a generator and a discriminator. The construction process of the virtual sound field environment generator includes: obtaining a plurality of training samples, each training sample containing random noise, the true audio frequency in the target sound field environment, and the environmental parameters in the target sound field environment; inputting the plurality of training samples into the virtual sound field environment generator, where the random noise and the environmental parameters in the target sound field environment are input into the generator, and the generator generates a simulated audio frequency, and the true audio frequency and the simulated audio frequency in the target sound field environment are input into the discriminator, and a corresponding probability value is output, and this probability value represents the probability that the input audio frequency is the true audio frequency; inputting the probability output when the discriminator inputs the true audio frequency and the probability output when the discriminator inputs the simulated audio frequency into the discriminant loss function to obtain the discriminant loss value; inputting the probability output when the simulated audio frequency is input into the generator loss function to obtain the generator loss value; according to the discriminant loss value and the generator loss value, calculating the partial derivatives of each parameter in the virtual sound field environment generator layer by layer, and updating the parameters until the preset requirements are met to obtain the trained virtual sound field environment generator.

[0041] Specifically, in the virtual sound field environment generator based on the adversarial network, a random noise vector and environmental parameters are used as the generator input. The generator combines the random noise vector and conditional information to generate simulated audio. The discriminator compares the simulated audio with the real audio in the target sound field environment to determine its authenticity, thereby providing feedback to the generator to prompt the generator to continuously improve the generation effect. After being processed by a series of neural network layers, the generator outputs simulated virtual sound field environment audio data. These data are significantly different from the real data in the initial stage of training, but as the training progresses, they will gradually approximate the characteristics of the real data to achieve the purpose of deceiving the discriminator. The discriminator judges the input real data and generated data and outputs a probability value indicating the probability that the input audio is real audio. For real audio, the goal of the discriminator is to output a probability value close to 1; for simulated audio, the goal of the discriminator is to output a probability value close to 0. Therefore, the generator and the discriminator are both adversarial and collaborative. In the continuous adversarial training process, the generator will strive to generate more realistic simulated audio to deceive the discriminator, while the discriminator will continuously improve its ability to distinguish between true and false data. During the training process, the discriminator will feedback the judgment result to the generator in the form of gradients, and the generator will adjust its network parameters according to these gradient information to make the generated data closer to the distribution of real audio data. After multiple rounds of training iterations, the generator gradually learns the distribution characteristics of real audio data, thereby being able to generate simulated audio data with increasingly high quality.

[0042] The specific training process includes: The goal of the generator is to maximize the output probability of the discriminator for the simulated audio. Therefore, its loss function can be constructed based on the output probability of the discriminator for the simulated audio. For example, it can be transformed into minimizing the negative log-likelihood, that is: , where, represents the output probability of the simulated audio in the discriminator module, represents the log-likelihood value of the output probability of the simulated audio in the discriminator module the expected value of, is the probability distribution of the simulated audio.

[0043] The discriminator should make the discrimination score of the real audio as high as possible compared to the discrimination probability of the simulated audio data. Therefore, the discriminator loss function can be: ; where, represents the input probability of the real audio in the discriminator module, represents the expectation of the input probability of the real audio in the discriminator module, represents the probability distribution of the real audio, represents the expectation of the output probability of the simulated audio in the discriminator module.

[0044] According to the above two loss functions, an overall loss function can be constructed. Specifically, add the two to obtain a total loss function value. Then, use the backpropagation algorithm to calculate the gradient of the loss function with respect to the discriminator parameters, and update the discriminator parameters according to the optimization algorithm (such as stochastic gradient descent, Adam, etc.) to make the discriminator more accurate in subsequent judgments.

[0045] When actually using this virtual sound field environment generator, the user can determine the corresponding virtual sound field environment generator according to the requirements. For example, if the audio integration system will be used in a concert hall, then the user can input the environmental requirement information, and then input the environmental information into the trained virtual sound field environment generator. The virtual sound field environment generator simulates the sound effect of the audio to be verified under this concert hall, that is, the simulated audio. Finally, collect this simulated audio and extract the performance parameters in the simulated audio. The performance parameters can include volume, frequency, signal-to-noise ratio, and so on.

[0046] The embodiment of the present invention provides an audio integration simulation prototype verification test platform. By constructing a virtual sound field environment generator, the audio effect of the audio integration system under different environmental requirements can be verified. Compared with the general sound field simulation method, the method proposed in this embodiment can more accurately verify the performance of the audio in a specific environment, ensure the effectiveness of the audio integration system in the actual target scenario, and can provide a reference for the user, which is beneficial for the user to adjust the internal parameters of the audio integration system on this basis.

[0047] As an optional implementation manner, the verification module includes: A user expectation input module, configured to receive the user's expectation for the audio signal output by the simulation prototype; An expected effect calculation module, configured to, based on reinforcement learning, regard the adjustment of the expected effect parameters of each audio processing module as the action of the intelligent agent, and measure the closeness between the audio signal output by the simulation prototype and the user's expectation according to a preset reward function to obtain the optimal expected effect of each audio processing module; A processing effect verification module, configured to calculate the difference between the output of the audio to be verified in each audio processing module of the simulation prototype and the expected effect of the corresponding audio processing module, and use the difference calculation result as the processing effect verification result of the audio processing module; A display module, configured to display the processing effect verification results of each audio processing module.

[0048] Exemplarily, in this embodiment, a method of using reinforcement learning is proposed to reverse-derive the optimal expected effects of each audio processing module from the results expected by the user. Specifically, first, the environment of reinforcement learning is defined, which includes a simulation prototype and the adjustable parameter ranges of each audio processing module. For example, for the equalizer module, the adjustable parameter may be the gain value of each frequency band, and its range is set from -20 dB to +20 dB; for the reverb module, the adjustable range of the reverb time may be from 0.1 s to 5 s, etc. Then, the agent is initialized, and the state of the agent consists of the current parameter settings of each audio processing module and the characteristics of the audio signal output by the simulation prototype. The reward function is used to measure the closeness of the audio signal output by the simulation prototype to the user's expectation. For example, for the volume expectation, the difference between the output volume and the user-expected volume can be calculated, and the smaller the difference, the higher the reward; for the sound quality expectation, by analyzing the difference between the frequency response of the output audio and the user-expected frequency response, methods such as mean square error are used to calculate the difference value, and the smaller the difference, the higher the reward. Considering multiple expectation factors comprehensively, corresponding weights are assigned to each factor to obtain the total reward value. For example, if the user pays more attention to the volume, a higher weight is assigned to the volume factor.

[0049] Specifically, the reward function can be: ; where represents the weight of the i-th metric, represents the reward of the i-th metric, and n represents the total number of metrics.

[0050] Taking the metrics including volume, timbre, and clarity as an example for illustration: ; where represents the reward value of volume, represents the maximum volume of the system, represents the user-expected volume, represents the actually output volume.

[0051] ; where represents the reward value of timbre, represents the user-expected audio spectrum, represents the actually output audio spectrum, and MSE represents the mean square error.

[0052] ; where represents the reward of clarity, represents the actually output signal-to-noise ratio, the maximum signal-to-noise ratio of the system.

[0053] The agent performs a series of actions in the environment (i.e., adjusts the expected effect parameters of the audio processing module). After each action, the simulation prototype processes a test audio signal according to the new parameter settings and outputs the results. The reward value for this action is calculated according to the reward function, and the agent updates the policy based on the reward value and the current state to select better actions. This can be achieved by using common reinforcement learning algorithms such as Q-learning, Deep Q-Network (DQN), etc. Repeat the above process for multiple rounds of training until the agent converges to a stable policy. At this time, the parameter settings of each audio processing module obtained are the optimal expected effects.

[0054] After obtaining the optimal parameters of each audio processing module, the output data of each audio processing module of the audio to be verified in the simulation prototype is obtained from the scenario tracking and scheduling module. These data include the time-domain waveform, frequency-domain characteristics, etc. of the audio. According to the different characteristics of the audio, a suitable difference calculation method is selected. For the time-domain waveform, the mean square error (MSE) can be calculated to measure the difference in amplitude between the output waveform and the expected waveform; for the frequency-domain characteristics, a spectral distance metric such as Kullback-Leibler divergence (KL-divergence) can be used to evaluate the difference between the output spectrum and the expected spectrum. For each audio processing module, considering the difference calculation results of multiple characteristics, corresponding weights are assigned to each characteristic to obtain a comprehensive difference value.

[0055] The embodiment of the present invention provides an audio integrated simulation prototype verification test platform, which uses the method of reinforcement learning to seek the optimal parameters of each audio processing module. Compared with manual determination, it not only saves manpower but also is more scientific.

[0056] As an optional implementation manner, an audio integrated simulation prototype verification test platform further includes: an integrated system configuration list modification module for receiving an instruction from the user to modify the integrated system configuration list.

[0057] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. An audio integrated simulation prototype verification test platform, characterized in that including: a user requirement input module for receiving user requirement information, including integrated system information and the audio to be verified; a simulation prototype building module for building a simulation prototype for processing the audio to be verified according to the integrated system information; a scenario tracking and scheduling module for using a timestamp and an event-driven mechanism to track and record in real time the inputs and outputs of the audio to be verified in each audio processing module in the simulation prototype; a verification module for verifying the processing effects of the audio to be verified in each audio processing module according to the inputs and outputs of the audio to be verified in each audio processing module in the simulation prototype.

2. The audio integrated simulation prototype verification test platform according to claim 1, characterized in that If the integrated system information is integrated system index information, the simulation prototype building module includes: an integrated system configuration search module for retrieving the corresponding integrated system configuration list according to the integrated system index information; a first simulation prototype building sub-module for building a simulation prototype for processing the audio to be verified according to the integrated system configuration list.

3. An audio integrated simulation prototype verification test platform according to claim 2, characterized in that The integrated system configuration list includes: audio function module numbers, audio function module sequences, audio function parameters, message middleware types, and hardware device parameters. Among them, each audio function module incorporates an audio processing algorithm, and the audio processing algorithms include a noise reduction algorithm, an equalizer algorithm, an audio encoding algorithm, a mixing algorithm, and an audio decoding algorithm.

4. The audio integrated simulation prototype verification test platform according to claim 1, characterized in that If the integrated system information is an audio integrated system construction document to be verified, the simulation prototype building module includes: a configuration factor reading module for reading the audio integrated system construction document to be verified and performing keyword recognition on the audio integrated system construction document to obtain integrated system configuration factors; a configuration factor matching module for performing fuzzy matching between the integrated system configuration factors and the names of the integrated system component sub-modules pre-stored in the platform; a configuration parameter determination module for extracting the corresponding parameters of the target integrated system configuration factors from the audio integrated system construction document to be verified, where the target integrated system configuration factors are the factors that match successfully with the names of the integrated system configuration lists pre-stored in the platform; a second simulation prototype building sub-module for building a simulation prototype for processing the audio to be verified according to the target integrated system configuration factors and their corresponding parameters.

5. The audio integrated simulation prototype verification test platform according to claim 1, characterized in that, When the integrated system information includes environmental requirement information, the verification module includes: a virtual sound field environment generator determination module for extracting environmental requirement information from the integrated system index information and determining the corresponding virtual sound field environment generator according to the environmental requirement information; a virtual sound field determination module for inputting the output audio of the simulation prototype into the corresponding virtual sound field environment generator to obtain a simulated audio; a virtual sound field performance determination module for collecting the simulated audio and extracting the performance parameters in the simulated audio.

6. The audio integrated simulation prototype verification test platform according to claim 5, characterized in that, The virtual sound field environment generator includes a generator and a discriminator. The construction process of the virtual sound field environment generator includes: obtaining a plurality of training samples, each of which contains random noise, the real audio in the target sound field environment, and the environmental parameters in the target sound field environment; Inputting multiple training samples into a virtual sound field environment generator, wherein random noise and environmental parameters in the target sound field environment are input into the generator, the generator generates simulated audio, and the real audio in the target sound field environment and the simulated audio are input into the discriminator, which outputs corresponding probability values, which represent the probability that the input audio is the real audio; The probability of the discriminator outputting real audio and the probability of the discriminator outputting simulated audio are input into the discriminant loss function to obtain the discriminant loss value; Input the probability of the output when the simulated audio is input into the generation loss function to obtain the generation loss value; According to the discrimination loss value and the generation loss value, the partial derivative of each parameter in the virtual sound field environment generator is calculated layer by layer, and the parameters are updated until the preset requirements are met to obtain a trained virtual sound field environment generator.

7. An audio integrated simulation prototype verification test platform according to claim 1, characterized in that Verification modules, including: A user expectation input module for receiving the user's expectation on the output audio signal of the simulation prototype; An expected effect calculation module is used to treat the adjustment of the expected effect parameters of each audio processing module as the action of the intelligent agent based on reinforcement learning, and to measure the degree of proximity between the audio signal output by the simulation prototype and the user's expectation based on a preset reward function to obtain the optimal expected effect of each audio processing module; a processing effect verification module, configured to calculate the difference between the output of each audio processing module of the audio to be verified in the simulation prototype and the expected effect of the corresponding audio processing module, and use the difference calculation result as the processing effect verification result of the audio processing module; The display module is used to display the processing effect verification results of each audio processing module.

8. An audio integrated simulation prototype verification test platform according to claim 3, characterized in that Also includes: The integrated system configuration list modification module is used to receive user instructions to modify the integrated system configuration list.

Images