Audio data generation method and device, electronic equipment and storage medium
By generating high-fidelity audio data through dynamic noise addition and convolution processing, the problem of reproducing the real acoustic environment in audio data in existing technologies is solved, the robustness of speech recognition algorithms is improved and the acquisition cost is reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-29
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies struggle to faithfully reproduce the complex characteristics of real acoustic environments when generating audio data, resulting in insufficient robustness of speech recognition algorithms in complex reverberation or dynamic interference scenarios, and high data acquisition costs.
By acquiring the audio generation request, determining the noise data and simulated distance, dynamically adding noise to the original audio data based on the noise data, and performing convolution processing using the target IR file, high-fidelity audio data is generated.
It effectively reproduces the complex characteristics of real acoustic environments, improves the scene realism of audio data, and reduces data acquisition costs.
Smart Images
Figure CN122369423A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to an audio data generation method, apparatus, electronic device, and storage medium. Background Technology
[0002] With the development of computer technology, voice interaction products such as smart speakers, in-vehicle voice systems, and mobile terminal assistants have been widely used in daily life and industrial scenarios. To ensure the robustness of speech recognition algorithms, a large amount of audio data covering different acoustic environments is needed to test the algorithms.
[0003] In related technologies, audio data can be obtained through manual collection. This method can acquire high-fidelity data; however, it relies on manual recording and annotation, which is time-consuming and labor-intensive. Alternatively, audio data can be synthesized based on fixed samples, such as superimposing clean speech with fixed noise samples to simulate a noisy acoustic environment. However, this method can only simulate changes in volume, and the generated audio data tends to have singular acoustic features, failing to reproduce the non-stationary characteristics of noise and complex spatial acoustic environments in real-world scenarios. Therefore, algorithms trained on such data have insufficient generalization ability, and their robustness will significantly decrease when applied to real-world scenarios containing complex reverberation or dynamic interference.
[0004] Due to the shortcomings of existing technologies, there is an urgent need for an audio data generation method that can reproduce the complex characteristics of real acoustic environments with high fidelity while reducing data acquisition costs.
[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] This disclosure provides an audio data generation method, apparatus, electronic device, and storage medium that, at least to a certain extent, reduces data acquisition costs while faithfully reproducing the complex characteristics of a real acoustic environment.
[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.
[0008] According to one aspect of this disclosure, an audio data generation method is provided, comprising: Obtain an audio generation request; determine noise data and simulated distance based on the audio generation request, wherein the noise data describes the noise required for the audio, and the simulated distance describes the distance between the sound source and the recording device; dynamically add noise to the original audio data based on the noise data to obtain corresponding noise-added frequency data; determine the corresponding target IR (Impulse Response) file based on the simulated distance; perform convolution processing on the noise-added frequency data based on the target IR file to obtain target audio data.
[0009] In some exemplary embodiments, the noise data includes a noise range, and the step of dynamically adding noise to the original audio data based on the noise data to obtain corresponding noise-added frequency data includes: determining an initial signal-to-noise ratio (SNR) value that satisfies the noise range; and dynamically adding noise to the original audio data according to the initial SNR value, wherein the initial SNR value is adjusted in real time within the noise range during the dynamic noise addition process to obtain the noise-added frequency data.
[0010] In some exemplary embodiments, the noise data includes a noise type, and the step of dynamically adding noise to the original audio data based on the noise data to obtain corresponding noise-added frequency data includes: determining at least one noise audio based on the noise type; and dynamically adding noise to the original audio data based on the at least one noise audio to obtain corresponding noise-added frequency data.
[0011] In some exemplary embodiments, the target IR file is used to describe at least one of the following: reverberation time, early reflection intensity, and direct sound attenuation of audio data at the simulated distance.
[0012] In some exemplary embodiments, determining the corresponding target IR file based on the simulated distance includes: determining the corresponding target IR file based on the simulated distance and spatial information, wherein the spatial information is used to describe at least one of the size of the space where the sound source is located and the wall material.
[0013] In some exemplary embodiments, after obtaining the audio generation request, the method further includes: determining the test scenario based on the audio generation request; obtaining at least one raw audio data under the test scenario, wherein any raw audio data contains a test speech under the test scenario.
[0014] In some exemplary embodiments, after performing convolution processing on the noise-enhanced audio data according to the target IR file to obtain target audio data, the method further includes: generating tag information corresponding to the target audio data, wherein the tag information is used to describe at least one of the original audio data corresponding to the target audio data, the target IR file, the noise audio, and the signal-to-noise ratio value.
[0015] According to another aspect of this disclosure, an audio data generation apparatus is also provided, comprising: The request retrieval module is used to retrieve audio generation requests; The first determining module is used to determine noise data and simulated distance based on the audio generation request. The noise data is used to describe the noise required for the audio, and the simulated distance is used to describe the distance between the sound source and the recording device. The dynamic noise addition module is used to dynamically add noise to the original audio data based on the noise data to obtain the corresponding noise-added audio data. The second determining module is used to determine the corresponding target IR file based on the simulated distance; The convolution module is used to perform convolution processing on the noise-added frequency data according to the target IR file to obtain the target audio data.
[0016] In some exemplary embodiments, the noise data includes a noise range, a dynamic noise addition module, configured to determine an initial signal-to-noise ratio (SNR) value that satisfies the noise range; and to dynamically add noise to the original audio data based on the initial SNR value, wherein the initial SNR value is adjusted in real time within the noise range during the dynamic noise addition process to obtain the noise-added audio data.
[0017] In some exemplary embodiments, the noise data includes a noise type and a dynamic noise addition module, which is used to determine at least one noise audio based on the noise type; and to dynamically add noise to the original audio data based on the at least one noise audio to obtain corresponding noise-added audio data.
[0018] In some exemplary embodiments, the target IR file is used to describe at least one of the following: reverberation time, early reflection intensity, and direct sound attenuation of audio data at the simulated distance.
[0019] In some exemplary embodiments, the second determining module is used to determine the corresponding target IR file based on the simulated distance and spatial information, wherein the spatial information is used to describe at least one of the size of the space where the sound source is located and the wall material.
[0020] In some exemplary embodiments, the apparatus provided in this disclosure further includes: The raw audio data acquisition module is used to determine the test scenario based on the audio generation request; and to acquire at least one raw audio data in the test scenario, wherein any raw audio data contains a test voice in the test scenario.
[0021] In some exemplary embodiments, the apparatus provided in this disclosure further includes: The tag information generation module is used to generate tag information corresponding to the target audio data. The tag information is used to describe at least one of the original audio data, the target IR file, noisy audio, and signal-to-noise ratio value corresponding to the target audio data.
[0022] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform an audio data generation method of any of the above via executing the executable instructions.
[0023] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the audio data generation method of any of the above.
[0024] According to another aspect of this disclosure, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the audio data generation method provided in various alternative embodiments of this disclosure.
[0025] The technical solutions provided in the embodiments of this disclosure can flexibly configure noise data and simulated distance based on audio generation requests. Therefore, they can apply specific noise to the original audio data according to the set noise data, and simulate the sound wave propagation characteristics based on the distance between the sound source and the receiving device. This can reproduce the complex characteristics of the real acoustic environment with high fidelity, effectively improve the scene realism of audio data generation, and reduce data acquisition costs.
[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0028] Figure 1 A schematic diagram of a system architecture according to an embodiment of this disclosure is shown; Figure 2 This diagram illustrates a flowchart of an audio data generation method according to an embodiment of the present disclosure; Figure 3 This diagram illustrates a flowchart of an audio data generation method according to an embodiment of the present disclosure; Figure 4 This diagram illustrates an audio data generation system according to an embodiment of the present disclosure. Figure 5 This diagram illustrates an audio data generation apparatus according to an embodiment of the present disclosure. Figure 6 This diagram illustrates a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 7 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation
[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0030] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0031] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.
[0032] Figure 1 A schematic diagram of an exemplary application system architecture to which the audio data generation method of the embodiments of this disclosure can be applied is shown. For example... Figure 1 As shown, the system architecture may include terminal device 101, network 102 and server 103.
[0033] In this process, terminal device 101 can send an audio generation request to server 103. Server 103 then receives the audio generation request; determines noise data and analog distance based on the request, whereby the noise data describes the noise required for the audio and the analog distance describes the distance between the sound source and the receiving device; dynamically adds noise to the original audio data based on the noise data to obtain corresponding noise-added frequency data; determines the corresponding target IR file based on the analog distance; and performs convolution processing on the noise-added frequency data based on the target IR file to obtain the target audio data. Finally, server 103 can return the target audio data to terminal device 101.
[0034] Alternatively, server 103 can send an audio generation request to terminal device 101. Terminal device 101 then receives the audio generation request; determines noise data and analog distance based on the request, whereby the noise data describes the noise required for the audio and the analog distance describes the distance between the sound source and the receiving device; dynamically adds noise to the original audio data based on the noise data to obtain corresponding noise-added frequency data; determines the corresponding target IR file based on the analog distance; and performs convolution processing on the noise-added frequency data based on the target IR file to obtain the target audio data. Finally, terminal device 101 can return the target audio data to server 103.
[0035] Network 102 is a medium used to provide a communication link between terminal device 101 and server 103, and can be a wired network or a wireless network.
[0036] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPSec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.
[0037] Terminal device 101 can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, wearable devices, augmented reality devices, virtual reality devices, etc.
[0038] Server 103 can be a server that provides various services, such as a backend management server that supports the device operated by the user using terminal device 101. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal device.
[0039] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.
[0040] Those skilled in the art will know that Figure 1The number of terminal devices 101, networks 102, and servers 103 shown is merely illustrative. Any number of terminal devices 101, networks 102, and servers 103 can be used as needed. This disclosure does not limit the number of such devices.
[0041] Under the above system architecture, this disclosure provides an audio data generation method, which can be executed by any electronic device with computing capabilities.
[0042] Figure 2 This diagram illustrates a flowchart of an audio data generation method according to an embodiment of the present disclosure, such as... Figure 2 As shown, the audio data generation method provided in this embodiment includes the following steps S202 to S210.
[0043] S202, Obtain audio generation request.
[0044] In an exemplary embodiment, the audio generation request can be used to request the generation of voice audio data used when conducting voice recognition tests on a voice interaction product.
[0045] For example, the audio generation request may include at least one acoustic parameter that describes the characteristics of speech audio data in a specific acoustic scene.
[0046] In some exemplary embodiments, after obtaining the audio generation request, the audio data generation method provided in this disclosure embodiment may further include: determining the test scenario based on the audio generation request; obtaining at least one original audio data in the test scenario, wherein any original audio data contains a test speech in the test scenario.
[0047] In some possible implementations, when a tester needs to initiate an audio generation request, the test scenario can be set up, so the audio generation request can include the test scenario, such as "open the curtains" or "open the car window".
[0048] Taking the scenario of "opening the curtains" as an example, at least one piece of raw audio data in this scenario can be obtained. For example, the raw audio data may include audio data of "opening the curtains" with different timbres, volumes, and speaking speeds, or it may include audio data of different expressions such as "open the curtains".
[0049] In an exemplary embodiment, raw audio data can be obtained from a raw speech library, which can be used to store pre-recorded, high-quality, clean speech samples, i.e., raw audio data. Each piece of raw audio data can contain corresponding basic tags, which may include information such as text content, speaker identification, and sampling rate.
[0050] S204, determine noise data and analog distance based on the audio generation request. The noise data is used to describe the noise required for the audio, and the analog distance is used to describe the distance between the sound source and the recording device.
[0051] In some possible implementations, the aforementioned acoustic parameters may include at least one of noise data and simulated distance. The noise data is used to describe at least one of the noise range and noise type required to generate the audio data.
[0052] In some possible implementations, when a tester needs to initiate an audio generation request, noise data and simulated distance can be set.
[0053] For example, the noise range can be used to describe the magnitude of the noise value, which can be represented, for example, based on the noise's signal-to-noise ratio, root mean square value, power, etc. The noise type can be distinguished based on, for example, the noise sampling scenario, which may include at least one of a shopping mall, a cafe, a vehicle interior, or a residential environment. Alternatively, it can be distinguished based on the noise content, for example, the noise content may include at least one of white noise, street noise, office conversations, and background music.
[0054] In an exemplary embodiment, the simulated distance is used to describe the distance between the sound source and the recording device, i.e., the sampling distance of the noise. It should be noted that different distances between the sound source and the recording device will result in differences in at least one of the following: reverberation time, early reflection intensity, and direct sound attenuation.
[0055] S206, dynamically adds noise to the original audio data based on the noise data to obtain the corresponding noise-added frequency data.
[0056] In some exemplary embodiments, noise data may include a noise range.
[0057] In this case, the original audio data is dynamically denoised based on the noise data to obtain the corresponding denoised frequency data, including: determining an initial signal-to-noise ratio (SNR) value that meets the noise range; dynamically denoising the original audio data according to the initial SNR value, wherein the initial SNR value is adjusted in real time within the noise range during the dynamic denoising process to obtain the denoised frequency data.
[0058] In an exemplary embodiment, the noise range can be represented as a range of signal-to-noise ratio (SNR) values from which noise is to be added. Therefore, an initial SNR value is first selected from the noise range.
[0059] In an exemplary implementation, when it is necessary to dynamically add noise to the original audio data, the effective power or amplitude of the original audio data can be calculated first, and then the power of the noise to be added can be determined based on the initial signal-to-noise ratio value. Then, the noise can be added based on the power value.
[0060] In one possible implementation, the power of the noise to be added can be calculated based on the following formula (1).
[0061]
[0062] In formula (1), N can represent the power of the noise to be added. S can represent the effective power of the original audio data. SNR (Signal-to-Noise Ratio) can represent the initial signal-to-noise ratio value.
[0063] In an exemplary embodiment, to achieve dynamic noise addition to the original audio data, the initial signal-to-noise ratio (SNR) value can be adjusted in real time within the noise range, resulting in a continuous set of SNR values. For example, if the SNR range is -5dB to 20dB, the initial SNR value can be set to 2dB, and this initial SNR value can be adjusted in 1dB increments to obtain a continuous set of SNR values. Then, the power of the corresponding noise to be added can be determined based on this continuous set of SNR values, thereby achieving dynamic noise addition.
[0064] In some exemplary embodiments, noise data may include noise type.
[0065] In this case, dynamic noise addition is performed on the original audio data based on the noise data to obtain the corresponding noise-added frequency data, including: determining at least one noise audio based on the noise type; and dynamically adding noise to the original audio data based on the at least one noise audio to obtain the corresponding noise-added frequency data.
[0066] In an exemplary embodiment, at least one type of noise audio can be obtained from a noise library, which can be used to classify and store recordings of various real-world environmental noises, i.e., noise audio. Furthermore, each noise audio in the noise library can contain a corresponding type label.
[0067] In some possible implementations, after determining the noise audio, the energy of the noise audio can be scaled to the power of the noise to be added, and then the noise audio can be mixed with the original audio data by adding sample points.
[0068] It should be noted that when more than one type of noise audio is acquired, the original audio data can be dynamically denoised based on each type of noise audio to obtain the denoised audio data corresponding to each noise audio. Alternatively, at least one type of noise audio can be mixed first, and then the original audio data can be dynamically denoised based on the mixed noise audio. This disclosure does not limit the scope of the embodiments.
[0069] S208 determines the corresponding target IR file based on the simulated distance.
[0070] In some exemplary embodiments, the target IR file is used to describe at least one of the following: reverberation time, early reflection intensity, and direct sound attenuation of audio data at an analog distance.
[0071] It should be noted that the target IR file records the process of sound being reflected in the corresponding space, absorbed by the walls, and eventually disappearing.
[0072] In an exemplary embodiment, an IR file matching the simulated distance can be retrieved from an IR library and used as the target IR file. Each IR file in the IR library may contain a distance label. Furthermore, this disclosure does not limit the number of target IR files.
[0073] In one possible implementation, taking a simulated distance of 3 meters as an example, an IR file with a distance label of 3 meters in the IR library can be used as the target IR file. Alternatively, if no IR file with a distance label of 3 meters is found in the IR library, at least one IR file with a distance label closest to 3 meters can be used as the target IR file.
[0074] In another possible implementation, the simulated distance can be represented as a simulated distance range, for example, the simulated distance can be 2 meters to 5 meters, then at least one target IR file can be determined from IR files whose distance labels are within the 2-meter to 5-meter range. Alternatively, when the simulated distance is represented as a simulated distance range, a simulated distance can be selected from the simulated distance range, and the target IR file can be determined based on the selected simulated distance.
[0075] In some exemplary embodiments, determining the corresponding target IR file based on the simulated distance includes: determining the corresponding target impulse response IR file based on the simulated distance and spatial information, wherein the spatial information is used to describe at least one of the size of the space where the sound source is located and the wall material.
[0076] In an exemplary embodiment, the audio generation request may include spatial information describing the space in which the scene to be tested is located. This spatial information may include, for example, the size of the space in which the sound source is located, the material of the walls in the space in which the sound source is located, and the reverberation value of the space in which the sound source is located.
[0077] In an exemplary embodiment, each IR file in the IR library may also contain a corresponding spatial tag to enable the acquisition of the target IR file based on simulated distance and spatial information.
[0078] S210: Perform convolution processing on the noise-added frequency data according to the target IR file to obtain the target audio data.
[0079] In an exemplary embodiment, when multiple target IR files are retrieved, each target IR file can be convolved with the noise-added frequency data to obtain the corresponding target audio data.
[0080] In one possible implementation, a convolution algorithm can be used to convolve the noise-added audio data with the target IR file. This convolution process mathematically simulates the physical process of sound waves propagating, reflecting, and superimposing within a corresponding distance and space. The output audio then possesses the spatial reverberation characteristics that should be present at that distance.
[0081] It should be noted that the noise-added frequency data can be used as the excitation signal, and the target IR file can be used as the system's impulse response. By performing a time-domain convolution operation between the excitation signal and the system's impulse response, a weighted superposition of the excitation signal's spectrum and the spatial transfer function is achieved. Based on the superposition principle, the noise-added frequency data is decomposed into an infinite number of time-shifted impulse components, and then linearly combined according to the weights defined in the target IR file. This allows for the accurate reproduction at the output of the multipath effect, frequency-selective fading, and energy attenuation characteristics of the sound wave propagating in that physical space.
[0082] In some exemplary embodiments, after convolving the noise-enhanced audio data according to the target IR file to obtain the target audio data, the method provided in this disclosure embodiment may further include: generating tag information corresponding to the target audio data, wherein the tag information is used to describe at least one of the original audio data, the target IR file, the noise audio, and the signal-to-noise ratio value corresponding to the target audio data.
[0083] In an exemplary embodiment, when generating target audio data, corresponding tag information can be generated. This tag information can be represented as a dynamic tag. The dynamic tag is used to record the lineage of the target audio data, that is, which original audio data the target audio data originated from, which noisy audio and IR file were used, and the specific signal-to-noise ratio value and analog distance.
[0084] In addition, the tag information may also include data version, search and usage records, etc.
[0085] The method provided in this disclosure can flexibly configure noise data and simulated distance based on audio generation requests. Therefore, it can apply specific noise to the original audio data according to the set noise data, and simulate the sound wave propagation characteristics based on the distance between the sound source and the receiving device. This can reproduce the complex characteristics of the real acoustic environment with high fidelity, effectively improve the scene realism of audio data generation, and reduce data acquisition costs.
[0086] Figure 3 This diagram illustrates a flowchart of an audio data generation method according to an embodiment of the present disclosure, such as... Figure 3 As shown, the audio data generation method provided in this embodiment includes the following steps S302 to S318.
[0087] S302, receive an audio generation request, the audio generation request including at least one of noise range, noise type, and analog distance range.
[0088] For example, when a tester needs to initiate a batch audio generation request through a Web (World Wide Web) interface or API (Application Programming Interface), they can set the parameters of the test scenario and specify at least one of the following: noise range, noise type, and simulated distance range. For instance, the noise range could be a signal-to-noise ratio of 5dB to 15dB, the simulated distance could be 1 meter to 3 meters, and the noise type could be highway noise.
[0089] S304, based on the audio generation request, select at least one clean raw audio data from the raw speech library.
[0090] In an exemplary embodiment, when two or more original audio data are selected from the original speech library, the following steps S306 to S316 can be performed on each original speech data to obtain the target audio data corresponding to each original speech data.
[0091] S306, for each piece of raw audio data, randomly or according to rules selects a specific signal-to-noise ratio value within the range of signal-to-noise ratio values.
[0092] For example, the corresponding precise parameters can be determined based on the noise range and the simulated distance range in the audio generation request. For example, if the signal-to-noise ratio range is 5dB to 15dB, 8.7dB can be randomly selected. In addition, if the simulated distance range is 1 meter to 3 meters, 2.2 meters can be randomly selected.
[0093] S308 inputs the selected signal-to-noise ratio value and the corresponding type of noise audio selected from the noise library into the dynamic noise addition module. The dynamic noise addition module is used to calculate and adjust the energy of the noise audio, mix it with the original audio data, and generate noise-added audio data.
[0094] S310: Select the target IR file corresponding to the simulated distance from the IR library.
[0095] It should be noted that the IR files in this IR library are pre-established through actual measurement or simulation, and the simulated distance corresponding to each IR file is clearly marked. The simulated distance can be, for example, 0.5 meters, 1 meter, 3 meters, 5 meters, etc.
[0096] S312 inputs the target IR file and noise-added audio data into the IR convolution module for convolution transport to obtain target audio data with a specific distance spatial sense.
[0097] S314, Generate the tag information for the target audio data.
[0098] For example, the tag information may include the original speech ID (Identification) corresponding to the original audio data used, the noise ID corresponding to the noise audio used, the noise type ID, the specific signal-to-noise ratio value used, the IR file ID corresponding to the target IR file, the analog distance, the generation timestamp, etc.
[0099] S316, store the generated target audio data and corresponding tag information in the data and tag warehouse.
[0100] It should be noted that the target audio data and its corresponding tag information can be stored as an inseparable record in the data and tag repository. Furthermore, the lineage between the target audio data and the original audio data, as well as the acoustic parameters used, can be established in the metadata and lifecycle management module.
[0101] S318, determine whether the audio generation request for the current batch has been completed.
[0102] If yes, then end. If no, return to S306.
[0103] This disclosure abstracts the generation of audio data used for testing into an automated process driven by key acoustic parameters (signal-to-noise ratio, distance). By using raw audio data, noise data, and IR files as basic materials, and enabling infinite combinations through parameter configuration, it achieves a highly efficient "record once, generate thousands of times" model.
[0104] Furthermore, in the audio data generation process, this embodiment introduces an IR library strongly correlated with the distance between the sound source and the receiving device, and dynamically selects the corresponding target IR file for convolution based on the configured simulated distance parameters. The IR files obtained from actual measurements at different propagation distances carry the unique physical characteristics of sound wave propagation in that distance scenario, thereby accurately reconstructing the actual acoustic environment through convolution operations. Therefore, this embodiment can effectively reproduce the core influence of spatial distance on the auditory characteristics of speech during the audio data generation process, significantly improving the scene realism of the generated audio data.
[0105] Finally, this embodiment of the disclosure achieves continuous and precise numerical control of the signal-to-noise ratio (SNR) value of the generated audio data, rather than limited level control, thereby enabling fine-grained and continuous setting of the SNR parameter. Simultaneously, during the audio data generation process, the system can automatically generate structured dynamic tags covering all precise generation parameters, which are strongly associated with the corresponding audio data. This design effectively avoids problems such as non-standard data labeling and difficulty in ensuring quality consistency from the data generation source, and facilitates subsequent data management and full-process traceability, ensuring that data parameters are searchable, traceable, and reusable.
[0106] Figure 4 A schematic diagram of an audio data generation system provided in an embodiment of this disclosure is shown, such as... Figure 4 As shown, the audio data generation system provided in this embodiment may include a resource layer, a core processing layer, a data interaction layer, and a storage and management layer.
[0107] The user interaction layer serves as the configuration interface for test tasks and scenarios, allowing users to input information such as the test scenario, noise range, and simulated distance. Based on this information, the user interaction layer then transmits the corresponding audio generation request to the core processing layer.
[0108] This core processing layer may include a dynamic data generation and management platform and a control and scheduling center. The dynamic data generation and management platform receives the audio generation request and selects raw audio data from the raw speech database based on the control and scheduling center's information. Additionally, the control and scheduling center can trigger internal processes, including a dynamic noise addition module and an IR convolution module.
[0109] In an exemplary embodiment, a noisy audio input to a dynamic noise-adding module can be obtained based on a noise library in the resource layer to obtain noise-added frequency data. Furthermore, a target IR file can be obtained based on an IR library. Then, the target IR file and the noise-added frequency data can be input into an IR convolution module to generate target audio data. Finally, corresponding dynamic tags can be generated based on the target audio data.
[0110] Finally, the generated target audio data and corresponding tag information can be stored in the storage and management layer. The data and tag repository stores the generated target audio data and corresponding tag information. The metadata and lifecycle management module is used for tracing the lineage of the target audio data, version management, and usage statistics.
[0111] In some embodiments, in addition to being based on pre-recorded measured IR files, the IR convolution module can be integrated into a parametric IR generator. This generator receives parameters such as distance, room size, and material reflectivity, and calculates the corresponding IR file in real time using a physical acoustic model (such as the mirror source method), then performs the appropriate convolution. Generating IR files through a generator improves the flexibility of IR file acquisition and is more suitable for simulating virtual environments that have not yet been measured.
[0112] In addition to energy scaling of fixed noise audio, the dynamic noise addition module in this embodiment can also perform dynamic noise addition based on a noise synthesis algorithm. For example, it can synthesize noise audio with specific spectral characteristics (such as powder noise, Brownian noise, etc.) in real time according to the noise type, and then mix the real-time synthesized noise audio with the original audio data, thereby reducing the dependence on a large noise library.
[0113] In one possible implementation, the audio data generation system can be encapsulated as an audio data generation service and provided externally through a corresponding interface. Therefore, testers or automated test scripts only need to send a configuration request in the appropriate format to asynchronously obtain download links for a batch of generated data, making it more suitable for cloud-native and continuous integration / continuous deployment environments. This disclosure effectively improves the efficiency of batch audio data generation.
[0114] It should be noted that the acquisition, storage, use, and processing of data in this disclosure comply with relevant regulations. The various types of data obtained in this disclosure, such as personal identity data, operational data, and behavioral data related to individuals, customers, and groups, have all been licensed.
[0115] Based on the same inventive concept, this disclosure also provides an audio data generation apparatus, as described in the following embodiments. Since the principle by which this apparatus solves the problem is similar to that of the method embodiments described above, the implementation of this apparatus embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be repeated.
[0116] Figure 5 This diagram illustrates an audio data generation apparatus according to an embodiment of the present disclosure, such as... Figure 5 As shown, the device includes: Request retrieval module 501 is used to retrieve audio generation requests; The first determining module 502 is used to determine noise data and analog distance according to the audio generation request. The noise data is used to describe the noise required for the audio, and the analog distance is used to describe the distance between the sound source and the recording device. The dynamic noise addition module 503 is used to dynamically add noise to the original audio data based on the noise data to obtain the corresponding noise-added frequency data. The second determining module 504 is used to determine the corresponding target IR file based on the simulated distance; The convolution module 505 is used to perform convolution processing on the noise-added frequency data according to the target IR file to obtain the target audio data.
[0117] In some exemplary embodiments, the noise data includes a noise range, and a dynamic noise addition module 503 is used to determine an initial signal-to-noise ratio (SNR) value that satisfies the noise range; and to dynamically add noise to the original audio data based on the initial SNR value, wherein the initial SNR value is adjusted in real time within the noise range during the dynamic noise addition process to obtain noise-added audio data.
[0118] In some exemplary embodiments, the noise data includes a noise type, and a dynamic noise addition module 503 is used to determine at least one noise audio based on the noise type; and to dynamically add noise to the original audio data based on the at least one noise audio to obtain corresponding noise-added audio data.
[0119] In some exemplary embodiments, the target IR file is used to describe at least one of the following: reverberation time, early reflection intensity, and direct sound attenuation of audio data at an analog distance.
[0120] In some exemplary embodiments, the second determining module 504 is used to determine the corresponding target IR file based on the simulated distance and spatial information, wherein the spatial information is used to describe at least one of the size of the space where the sound source is located and the wall material.
[0121] In some exemplary embodiments, the apparatus provided in this disclosure further includes: The raw audio data acquisition module is used to determine the test scenario based on the audio generation request; and to acquire at least one raw audio data in the test scenario, wherein any raw audio data contains a test voice in the test scenario.
[0122] In some exemplary embodiments, the apparatus provided in this disclosure further includes: The tag information generation module is used to generate tag information corresponding to the target audio data. The tag information is used to describe at least one of the following: the original audio data, the target IR file, the noisy audio, and the signal-to-noise ratio value.
[0123] It should be noted that the above-mentioned request acquisition module 501, first determination module 502, dynamic noise addition module 503, second determination module 504, and convolution module 505 correspond to S202 to S210 in the method embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the apparatus, can be executed in a computer system such as a set of computer-executable instructions.
[0124] The apparatus provided in this disclosure can flexibly configure noise data and simulated distance based on audio generation requests. Therefore, it can apply specific noise to the original audio data according to the set noise data, and simulate the sound wave propagation characteristics based on the distance between the sound source and the receiving device. This can reproduce the complex characteristics of the real acoustic environment with high fidelity, effectively improve the scene realism of audio data generation, and reduce data acquisition costs.
[0125] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."
[0126] This disclosure provides an electronic device, which, exemplarily, includes a processor and a memory. The memory can be used to store executable instructions of the processor. The processor is configured to perform the audio data generation method provided in this disclosure by executing the executable instructions.
[0127] The following reference Figure 6 To describe an electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.
[0128] like Figure 6 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including storage unit 620 and processing unit 610).
[0129] The storage unit stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 610 can perform the following steps of the above method embodiments: Obtain the audio generation request; determine the noise data and simulated distance based on the audio generation request. The noise data is used to describe the noise required for the audio, and the simulated distance is used to describe the distance between the sound source and the receiving device; dynamically add noise to the original audio data based on the noise data to obtain the corresponding noise-added frequency data; determine the corresponding target IR file based on the simulated distance; perform convolution processing on the noise-added frequency data based on the target IR file to obtain the target audio data.
[0130] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.
[0131] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0132] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0133] Electronic device 600 can also communicate with one or more external devices 640 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0134] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.
[0135] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described audio data generation method.
[0136] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored. When executed by a processor, the computer program can implement the audio data generation method provided in the embodiments of this disclosure. The computer-readable storage medium may be a readable signal medium or a readable storage medium.
[0137] Figure 7 This illustration shows a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure, such as... Figure 7As shown, the computer-readable storage medium 700 stores a program product capable of implementing the methods described above. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.
[0138] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0139] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.
[0140] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.
[0141] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0142] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0143] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.
[0144] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.
[0145] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope of this disclosure is indicated by the appended claims.
Claims
1. A method for generating audio data, characterized in that, include: Get the audio generation request; Based on the audio generation request, noise data and analog distance are determined. The noise data is used to describe the noise required for the audio, and the analog distance is used to describe the distance between the sound source and the recording device. Based on the noise data, the original audio data is dynamically noise-added to obtain the corresponding noise-added frequency data; The corresponding target impulse response IR file is determined based on the simulated distance; The target audio data is obtained by convolving the noise-added frequency data according to the target IR file.
2. The audio data generation method according to claim 1, characterized in that, The noise data includes a noise range, and the step of dynamically adding noise to the original audio data based on the noise data to obtain the corresponding noise-added frequency data includes: Determine an initial signal-to-noise ratio value that satisfies the noise range; The original audio data is dynamically noise-added based on the initial signal-to-noise ratio (SNR) value, wherein the initial SNR value is adjusted in real time within the noise range during the dynamic noise-addition process to obtain the noise-added audio data.
3. The audio data generation method according to claim 1, characterized in that, The noise data includes noise types, and the step of dynamically adding noise to the original audio data based on the noise data to obtain corresponding noise-added frequency data includes: Determine at least one noise audio based on the noise type; The original audio data is dynamically noise-added based on at least one noisy audio signal to obtain the corresponding noise-added audio signal data.
4. The audio data generation method according to any one of claims 1 to 3, wherein the target IR file is used to describe at least one of the reverberation time, early reflection intensity, and direct sound attenuation of the audio data at the simulated distance.
5. The audio data generation method according to claim 4, characterized in that, The step of determining the corresponding target IR file based on the simulated distance includes: Based on the simulated distance and spatial information, the corresponding target IR file is determined, wherein the spatial information is used to describe at least one of the following: the size of the space where the sound source is located and the wall material.
6. The audio data generation method according to any one of claims 1 to 3, characterized in that, After obtaining the audio generation request, the process also includes: The test scenario is determined based on the audio generation request; Obtain at least one raw audio data set for the scenario to be tested, wherein any raw audio data set contains a test speech set for the scenario to be tested.
7. The audio data generation method according to any one of claims 1 to 3, characterized in that, After performing convolution processing on the noise-added frequency data according to the target IR file to obtain the target audio data, the method further includes: Generate tag information corresponding to the target audio data. The tag information is used to describe at least one of the original audio data, the target IR file, noisy audio, and signal-to-noise ratio value corresponding to the target audio data.
8. An audio data generation device, characterized in that, include: The request retrieval module is used to retrieve audio generation requests; The first determining module is used to determine noise data and simulated distance based on the audio generation request. The noise data is used to describe the noise required for the audio, and the simulated distance is used to describe the distance between the sound source and the recording device. The dynamic noise addition module is used to dynamically add noise to the original audio data based on the noise data to obtain the corresponding noise-added audio data. The second determining module is used to determine the corresponding target IR file based on the simulated distance; The convolution module is used to perform convolution processing on the noise-added frequency data according to the target IR file to obtain the target audio data.
9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the audio data generation method according to any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the audio data generation method according to any one of claims 1 to 7.
11. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, wherein a processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the electronic device to perform the audio data generation method as described in any one of claims 1 to 7.