Audio processing method and device, electronic equipment, storage medium and vehicle

Abstract

The invention relates to an audio processing method and device, electronic equipment, a storage medium and a vehicle, and particularly relates to the technical field of audio processing. The method comprises the following steps: acquiring user voice, and identifying text content corresponding to the user voice; the text content is input into a term conversion model, target audio terms corresponding to the text content and output by the term conversion model are obtained, the term conversion model is obtained through training on the basis of the natural voice text and sample annotation data, and the sample annotation data are audio terms annotated on the basis of the natural voice text; querying a target audio parameter corresponding to the target audio term according to a preset mapping relationship between the audio parameter and the audio term; and processing the original audio according to the target audio parameter. The embodiment of the invention is used for solving the problems that the sound effect adjustment operation is difficult and the mode is fixed.

Description

Technical Field

[0001] The present disclosure relates to the field of assisted driving, and in particular to an audio processing method, device, electronic device, storage medium and vehicle. Background Art

[0002] At present, the sound effects of audio playback devices are mainly adjusted by adjusting the audio parameters of the equalizer, or by selecting from the sound effect modes pre-made by the service provider. The various audio parameters of the equalizer are relatively professional, which makes it difficult for non-professionals to adjust the sound effects. Although the sound effect modes pre-made by the service provider are relatively simple to operate, they are difficult to meet the personalized and diversified auditory needs of users. Therefore, there is an urgent need for a sound effect adjustment method that supports personalized customization and is easy to operate. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an audio processing method, device, electronic device, storage medium and vehicle, which can adjust the sound effects through natural voice control, with simple operation and diversified effects.

[0004] In order to achieve the above objectives, the technical solutions provided by the embodiments of the present disclosure are as follows:

[0005] In a first aspect, the present disclosure provides an audio processing method, comprising:

[0006] Acquire user voice and identify text content corresponding to the user voice;

[0007] Inputting the text content into a term conversion model to obtain a target audio term corresponding to the text content output by the term conversion model, wherein the term conversion model is trained based on natural speech text and sample annotated data, and the sample annotated data is audio terms annotated based on the natural speech text;

[0008] According to a preset mapping relationship between audio parameters and audio terms, querying a target audio parameter corresponding to the target audio term;

[0009] The original audio is processed according to the target audio parameter.

[0010] As any optional implementation manner of the embodiments of the present disclosure, the training method of the term conversion model includes:

[0011] Acquire natural speech text, and annotate audio terms based on the natural speech text to obtain the sample annotated data;

[0012] Inputting the natural speech text into an initial model to obtain a predicted audio term output by the initial model;

[0013] Calculating a loss function value according to the sample annotation data, the predicted audio term and a preset loss function;

[0014] If the loss function value is greater than the loss threshold, the model parameters of the initial model are adjusted until the loss function value is less than or equal to the loss threshold, thereby obtaining a converged term conversion model.

[0015] As any optional implementation of the embodiments of the present disclosure, the audio terms include: at least one of instrument frequency band terms, note frequency band terms, part-of-speech gain terms and sound position information.

[0016] As any optional implementation of the embodiments of the present disclosure, the target audio terms include: instrument frequency band terms, part-of-speech gain terms and sound position information;

[0017] The target audio parameters include: the frequency range corresponding to the instrument frequency band term, the gain value corresponding to the part-of-speech gain term, and the sound identification corresponding to the sound position information;

[0018] The processing of the original audio according to the audio parameters includes:

[0019] According to the gain value corresponding to the part-of-speech gain term, the audio segment involved in the frequency range corresponding to the instrument frequency band term in the original audio is adjusted to obtain the target audio;

[0020] The target audio is sent to the speaker indicated by the speaker identifier.

[0021] As any optional implementation manner of the embodiments of the present disclosure, the inputting the text content into a term conversion model to obtain a target audio term corresponding to the text content output by the term conversion model includes:

[0022] Extracting target keywords from the text content, the target keywords including at least one of musical instrument keywords, note keywords, part of speech keywords, and sound location keywords;

[0023] The target keyword is input into the term conversion model to obtain a target audio term corresponding to the target keyword output by the term conversion model.

[0024] As any optional implementation manner in the embodiments of the present disclosure, the method further includes:

[0025] Sending the processed target audio to a corresponding speaker so that the speaker plays the target audio;

[0026] If a new user voice is acquired within the preset time period, extracting a first keyword from the new user voice;

[0027] When the first keyword matches the target keyword, the target audio is processed according to the target audio parameter to obtain a first audio;

[0028] The first audio is sent to the speaker so that the speaker plays the first audio.

[0029] As any optional implementation manner of the embodiments of the present disclosure, the acquiring of user voice and identifying text content corresponding to the user voice includes:

[0030] Preprocessing the acquired user voice, wherein the preprocessing includes denoising and voice extraction;

[0031] The preprocessed user voice is recognized to obtain text content corresponding to the user voice.

[0032] In a second aspect, the present disclosure provides an audio processing device, comprising:

[0033] An acquisition module, used to acquire user voice and identify text content corresponding to the user voice;

[0034] A term conversion module, used for inputting the text content into a term conversion model to obtain a target audio term corresponding to the text content output by the term conversion model, wherein the term conversion model is trained based on natural speech text and sample annotation data, and the sample annotation data is audio terms annotated on the basis of the natural speech text;

[0035] A query module, configured to query a target audio parameter corresponding to the target audio term according to a preset mapping relationship between the audio parameter and the audio term;

[0036] The processing module is used to process the original audio according to the target audio parameter.

[0037] As any optional implementation in the embodiments of the present disclosure, the term conversion module is further used to train the term conversion model, including: obtaining natural speech text, and annotating audio terms based on the natural speech text to obtain the sample annotation data;

[0038] Inputting the natural speech text into an initial model to obtain a predicted audio term output by the initial model;

[0039] Calculating a loss function value according to the sample annotation data, the predicted audio term and a preset loss function;

[0040] If the loss function value is greater than the loss threshold, the model parameters of the initial model are adjusted until the loss function value is less than or equal to the loss threshold, thereby obtaining a converged term conversion model.

[0041] As any optional implementation of the embodiments of the present disclosure, the audio terms include: at least one of instrument frequency band terms, note frequency band terms, part-of-speech gain terms and sound position information.

[0042] As any optional implementation of the embodiments of the present disclosure, the target audio terms include: instrument frequency band terms, part-of-speech gain terms and sound position information;

[0043] The audio parameters include: the frequency range corresponding to the instrument frequency band term, the gain value corresponding to the part-of-speech gain term and the sound identification corresponding to the sound position information;

[0044] The processing module is specifically used to: adjust the audio segment involved in the frequency range corresponding to the instrument frequency band term in the original audio according to the gain value corresponding to the part-of-speech gain term to obtain the target audio; and send the target audio to the speaker indicated by the speaker identifier.

[0045] As any optional implementation of the embodiments of the present disclosure, the term conversion module is specifically used to extract target keywords from the text content, and the target keywords include at least one of musical instrument keywords, note keywords, part of speech keywords, and sound location keywords;

[0046] The target keyword is input into the term conversion model to obtain a target audio term corresponding to the target keyword output by the term conversion model.

[0047] As any optional implementation manner of the embodiments of the present disclosure, the processing module, after processing the original video according to the target audio parameter, is further used to:

[0048] Sending the processed target audio to the corresponding speaker so that the speaker plays the target audio;

[0049] If a new user voice is acquired within the preset time period, extracting a first keyword from the new user voice;

[0050] When the first keyword matches the target keyword, processing the target audio according to the target audio parameter to obtain a first audio;

[0051] The first audio is sent to the speaker, so that the speaker plays the first audio.

[0052] As any optional implementation of the embodiments of the present disclosure, the acquisition module is specifically used to pre-process the acquired user voice, and the pre-processing includes denoising and voice extraction;

[0053] The preprocessed user voice is recognized to obtain text content corresponding to the user voice.

[0054] In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the audio processing method as described in the first aspect or any optional embodiment thereof.

[0055] In a fourth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the audio processing method as described in the first aspect or any optional embodiment thereof is implemented.

[0056] In a fifth aspect, the present disclosure provides a vehicle, comprising: a stereo, and an audio processing device as described in the second aspect or any optional implementation manner thereof, or an electronic device as described in the third aspect.

[0057] In a sixth aspect, the present disclosure provides a computer program product, comprising: when the computer program product is run on a computer, the computer is enabled to implement the audio processing method as described in the first aspect or any optional implementation manner thereof.

[0058] The audio processing method provided by the embodiment of the present disclosure first acquires user voice, recognizes the user voice to obtain the corresponding text content, and then converts the text content in natural language form into target audio terms through a term conversion model, and the term conversion model is pre-trained based on natural voice text and sample annotation data, wherein the sample annotation data is audio terms annotated on the basis of natural voice text; then, based on the mapping relationship between preset audio parameters and audio terms, the target audio parameters corresponding to the target audio terms are queried to process the original audio according to the target audio parameters, so that ordinary users can use natural voice to control sound effect adjustment and realize personalized sound effect adjustment; the voice control operation is simple, and the user's natural voice is converted into target audio terms that can be understood by the machine, and accurately adjusted to the sound effect expected by the user, which can meet the diversified needs of users and enhance the user's auditory experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 A flowchart of an audio processing method provided by an embodiment of the present disclosure;

[0062] Figure 2 A structural schematic diagram of an audio processing device is provided for an embodiment of the present disclosure;

[0063] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0064] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0065] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0066] The audio processing method provided in the embodiments of the present disclosure may be implemented by an audio processing device or an electronic device, wherein the electronic device may include but is not limited to: a car machine, a server, a personal computer, a laptop computer, a tablet computer, a smart phone, etc. Car machine. The operating system of the car machine may include Android (Android), a mobile operating system (iOS) developed by Apple, an operating system (Windows) developed by Microsoft Corporation of the United States, etc., and the embodiments of the present disclosure are not limited to this. The server includes but is not limited to a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of computers or network servers in cloud computing, wherein cloud computing is a kind of distributed computing, a super virtual computer consisting of a group of loosely coupled computer sets. Among them, the electronic device can be operated alone to implement the present disclosure, or it can be connected to the network and implement the present disclosure through interactive operations with other electronic devices in the network. Among them, the network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN) network, etc.

[0067] It should be noted that the protection scope of the audio processing method described in the embodiment of the present disclosure is not limited to the execution order of the steps listed in the present embodiment, and all solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present disclosure are included in the protection scope of the present disclosure.

[0068] like Figure 1 As shown, Figure 1 The present invention provides a flowchart of an audio processing method, which includes the following steps S101 to S104:

[0069] S101: Acquire user voice and identify text content corresponding to the user voice.

[0070] Among them, user voice refers to the natural voice of the user speaking normally. In some embodiments, the method can be applied to the audio system, and the audio system collects the user voice through a configured sound collector, such as a microphone, a microphone array, etc. In the actual environment, the collection of user voice is not completely non-interrupting. Therefore, after the user voice is collected, the user voice is pre-processed, including but not limited to: denoising, human voice extraction, so as to remove invalid information such as noise in the user voice, extract the effective human voice segment in the user voice, and improve the accuracy of subsequent voice recognition.

[0071] In some embodiments, the user's speech is converted into text to obtain text content corresponding to the user's speech. Automatic speech recognition technology (ASR, Automatic Speech Recognition): a technology that converts human speech into text, recognizes and decodes the features extracted from the speech, and converts the user's speech into text. ASR is mainly based on a neural network model, including but not limited to an acoustic model and a language model. In some embodiments, after the user's speech is converted into text content, target keywords are extracted from the text content, and the target keywords include but are not limited to at least one of the following: instrument keywords, note keywords, part of speech keywords, sound orientation keywords, etc.

[0072] For example, instrument keywords include: drum tune, violin, piano, etc.; note keywords include: treble, bass, middle, etc.; part-of-speech keywords are divided into adjectives and verbs, including strengthen, weaken, soft, heavier, etc.; sound direction keywords include front row, left side, right side, back row, etc. The present disclosure is not limited to this.

[0073] The above embodiment extracts target keywords after voice-to-text conversion, thereby reducing the amount of data processed in the subsequent term conversion process and improving efficiency. The target keywords are effective information in the user's voice, and at the same time improves the accuracy of term conversion.

[0074] S102: Input the text content into a term conversion model to obtain a target audio term corresponding to the text content output by the term conversion model.

[0075] The term conversion model is pre-trained based on natural speech text and sample annotated data. The sample annotated data is audio terms annotated based on natural speech text.

[0076] Audio terms refer to professional audio terms, or customized audio terms. Professional audio terms are recognized audio terms in the music field and have a certain degree of professionalism; customized audio terms can be set according to actual needs. For example, in the field of car audio, customized audio terms can be front door panel speakers, etc. Audio terms include but are not limited to at least one of the following: instrument frequency band terms, note frequency band terms, part-of-speech gain terms, audio position information, etc. Exemplarily, the correspondence between natural speech text and audio terms is shown in Table 1:

[0077] Table 1

[0078]

[0079] As shown in Table 1 above, the sample annotation data is based on the natural speech text to mark the type and content of the audio term. It should be noted that the above table only exemplifies the partial correspondence between the natural speech text and the audio term, where the audio position information can be used to indicate the orientation information of the audio, for example, the front row-audio position information-A position audio, the A position audio is the audio that is relatively located in the front row in the multi-audio scene, where the orientation information of the audio can be set, for example, by setting the orientation information of the audio through the buttons on the audio, or, in the vehicle audio scene, the orientation information of each audio is pre-configured by the manufacturer, for example, the A position audio refers to the front door panel speaker in the vehicle cabin; the middle-audio position information-B position audio is the audio that is relatively located in the middle in the multi-audio scene, for example, the B position audio refers to the center speaker in the vehicle cabin.

[0080] The following will illustrate the training process of the term conversion model:

[0081] (1) Obtaining training samples

[0082] Training samples refer to natural speech text and sample annotation data. Natural speech text can be crawled from the database, and sample annotation data can be manually annotated or automatically annotated by intelligent algorithms, which annotates audio terms based on natural speech text.

[0083] For example, assuming a natural speech text is "the sound of the violin is louder", it is annotated and the sample annotation data obtained is: the sound of the violin (violin) is louder (adjective, gain value increases).

[0084] In some embodiments, after obtaining the sample annotated data, a random masking process is performed on the sample annotated data, that is, the audio terms that have been annotated in the natural speech text are randomly masked to obtain masked data. The masked data can be used as a verification set to verify the trained model.

[0085] Exemplarily, for the sample labeled data: the sound of the violin (violin) is louder (adjective, the gain value increases), random masking is performed, and the masked data obtained is the sound of the violin (mask) is louder (adjective, the gain value increases).

[0086] (2) Constructing the initial model

[0087] In some embodiments, the initial model is constructed based on Natural Language Understanding (NLU), or Transformer (translation) model, or Bidirectional Encoder Representations from Transformer (BERT) model, wherein the Transformer model is a neural network model based on an attention mechanism, which extracts and analyzes natural language text through the attention mechanism. This disclosure is not limited to this.

[0088] (3) Model training

[0089] The natural speech text is input into the initial model, and the initial model processes the natural speech text to output predicted audio terms, wherein the predicted audio terms include but are not limited to at least one of the following: predicted instrument frequency band terms, predicted note frequency band terms, predicted part-of-speech gain terms, and predicted sound position information.

[0090] There is a preset loss function for model training, such as a cross entropy loss function, which is not limited in the present disclosure. The loss function value is calculated based on the sample annotation data, predicted professional terms, and the preset loss function, and the loss function value is compared with the loss threshold. If the loss function value is greater than the loss threshold, it means that the model has not converged, and the model parameters of the initial model are adjusted. Then the natural speech text is re-input into the initial model after the model parameters are adjusted, and the loss function value is calculated and compared until the loss function value is less than or equal to the loss threshold, indicating that the model has converged, and the converged model is used as the term conversion model.

[0091] In some embodiments, during the model training process, mask data is used for verification, the mask data is input into the initial model, and the model accuracy is verified to be less than or equal to the preset accuracy based on the predicted audio terms and mask data output by the model. If so, the trained term conversion model is applied to the scenario of voice-controlled sound effect adjustment. If not, the hyperparameters of the model are adjusted by feedback until the calculated accuracy is less than or equal to the preset accuracy.

[0092] The above-mentioned embodiment obtains a term conversion model based on natural language text and sample annotation data training, which can be used for audio processing to realize natural voice control and adjustment of sound effects. The text content is used as the input of the term conversion model to obtain the target audio terms corresponding to the text content output by the term conversion model, including at least one of instrument frequency band terms, note frequency band terms, part-of-speech gain terms and sound position information.

[0093] For example, assuming that the text content corresponding to the user's voice is: adjust the drum tone in the front row a little heavier, the text content is input into the trained term conversion model, and the output target audio terms include: sound position information (speaker A, such as the front door panel speaker), instrument segment term (drum), part-of-speech gain term (adjective, gain value increases).

[0094] S103: Query the audio parameters corresponding to the target audio term according to the preset mapping relationship between the audio parameters and the audio term.

[0095] The present disclosure pre-sets the mapping relationship between audio terms and audio parameters, and stores the mapping relationship in a database.

[0096] In some embodiments, the audio parameter corresponding to the instrument frequency band term is a first frequency range, and the audio parameter corresponding to the note frequency band term is a second frequency range.

[0097] Exemplarily, as shown in Table 2, Table 2 shows different audio parameters corresponding to different audio professional terms.

[0098] Table 2

[0099]

[0100]

[0101] Table 2 above is only an exemplary description, and the present disclosure does not limit the specific data and format of the audio parameters. The audio parameters support user customization.

[0102] It should be noted that the audio position information is pre-set information that characterizes the relative position of the audio, and the audio identifier corresponding to the audio position information is also pre-set. The audio identifier is used to uniquely identify the audio, and different audio identifiers indicate different audio. The audio indicated by different audio position information may be the same or different. For example, assuming that the audio at position A is the front row audio, the audio identifier a is the identifier of the front door panel speaker; assuming that the audio at position D (not shown in the table) is the driving area audio, the audio identifier corresponding to the audio at position D includes the identifier of the front door panel speaker, the identifier of the center speaker, and the identifier of the subwoofer. The present disclosure does not limit the number of audio identifiers corresponding to the audio position information.

[0103] Using the previous example, assuming that the text content corresponding to the user's voice is: adjust the drum tone in the front row a little heavier, the target audio terms corresponding to the text content include: speaker position information (A position sound), instrument segment term (drum), part-of-speech gain term (adjective, gain value increased), and the target audio parameters corresponding to the target audio terms obtained by querying based on Table 2 above include: A position sound corresponds to sound identifier a; drum corresponds to 30Hz~300HZ; adjective, gain value increased corresponds to +1dB. In this way, the user's natural voice is processed to obtain the target audio parameters that can adjust the sound effect.

[0104] S104: Process the original audio according to the target audio parameters.

[0105] In some embodiments, the target audio term includes: a musical instrument frequency band term, a part-of-speech gain term, and sound location information. Correspondingly, the target audio parameter includes: a frequency range corresponding to the musical instrument frequency band term, a gain value corresponding to the part-of-speech gain term, and a sound identification corresponding to the sound location information. The original audio is processed according to the target audio parameter. First, the audio segment involved in the frequency range corresponding to the musical instrument frequency band term in the original audio is adjusted according to the gain value corresponding to the part-of-speech gain term to obtain the target audio, and then the target audio is sent to the sound indicated by the sound identification so that the sound can receive the adjusted target audio and play it, thereby adjusting the sound effect through natural voice control.

[0106] For example, the user voice is "adjust the drum tone of the front row a little heavier", after executing the above steps S101 to S103, the target audio parameters are obtained: audio identifier a (front door panel speaker), 30Hz~300Hz, +1dB, and then according to the target audio parameters, the gain of the audio segment at 30Hz~300Hz in the original audio is increased by 1dB according to the gain value, and sent to the front door panel speaker for output. That is, the gain of the audio segment to be output by the front door panel speaker at 30Hz~300Hz is increased by 1dB before output.

[0107] In some embodiments, if a new user voice is obtained within a preset time after executing step S104, an association relationship between the user voice and the historical user voice can be established. The historical user voice here refers to the user voice obtained in step S101. The user voice obtained after the preset time can be defined as the historical user voice. It can be understood that in the actual process of adjusting the sound effect, the user needs to speak multiple times for voice control in order to adjust to the desired audio effect. In order to facilitate the user's voice control, the user does not need to speak the part that needs to be adjusted multiple times and completely. Optionally, after the original audio is processed according to the target audio parameters, the target audio is obtained, and the target audio is sent to the corresponding speaker so that the speaker plays the target audio. Then, it can be detected whether there is a new user voice within the preset time length. If a new user voice is obtained within the preset time length, the first keyword is extracted from the new user voice, and the first keyword is compared with the target keyword to see whether it matches. Specifically, the similarity between the first keyword and the target keyword is calculated to determine whether the first keyword is the same as or similar to the target keyword, which can be understood as determining whether the first keyword and the target keyword are synonyms or near synonyms; thus, when the first keyword matches the target keyword, that is, when the first keyword is a synonym or near synonym of the target keyword, the aforementioned target audio is processed according to the target audio parameters obtained based on the target keyword to obtain the first audio, and then the first audio is sent to the same speaker so that the speaker plays the first audio, and the sound effect of the first audio is changed compared with the target audio, thereby realizing multiple and simple adjustment of the sound effect through voice. .

[0108] For example, assuming the historical user voice is "adjust the front row drum tone a little heavier", and the new user voice is "a little heavier", it indicates that the user hopes to increase the gain of the audio segment between 30Hz and 300Hz in the original audio by 1dB based on the previous adjustment, and transmit it to the front door panel speaker for output. However, the user does not need to say the complete sentence: "Adjust the tone of the front drum a little louder". The user only needs to say "a little louder" within 3 seconds. Based on the new user voice, the first keyword "a little louder" is extracted, which is the same as the target keyword "a little louder" extracted before. Then, according to the target audio parameters corresponding to the previous target keyword: audio identifier a (front door panel speaker), 30Hz~300Hz, +1dB, based on the last processing result, the gain of the audio segment at 30Hz~300Hz in the original audio is increased again by 1dB according to the gain value to obtain the first audio, and the first audio is sent to the front door panel speaker for output. Compared with the original audio, the gain of the audio segment at 30Hz~300Hz of the first audio output by the front door panel speaker is increased by 2dB. This makes voice-controlled sound effect adjustment more humane and improves the user's audio-visual experience. It should be emphasized that the present disclosure does not limit the number of times a new user voice is obtained. The user can speak a new user voice again after a new round of sound effect adjustment, that is, the user can speak "a little heavier" and other brief voices containing target keywords multiple times, thereby controlling the sound effect through voice control multiple times until the desired effect is achieved. Following the previous example, after the user adjusts the audio effect output by the speaker of the front door panel through voice "a little heavier", if he is still not satisfied, he can continue to speak similar sentences "a little heavier" to control the sound effect to further adjust until the desired audio effect is achieved.

[0109] In the above embodiment, after executing step S104, user voice detection is performed within a preset time length. If a new user voice is detected within the preset time length, an association relationship is established between the new user voice and the historical user voice, and then the corresponding text content of the new user voice is identified, and the first keyword is extracted from it. The first keyword is compared with the previous target keyword to see if it matches. If it matches, the audio is processed according to the target audio parameters corresponding to the target keyword, so as to further adjust the audio sound effects based on the target audio parameters corresponding to the historical user voice according to the aforementioned association relationship until the audio effect expected by the user is adjusted. If it does not match, it means that the user has made different adjustments to the audio than before, and a new round of term conversion process is started, and the term conversion model is used to convert the text content into target audio terms to determine the corresponding new audio parameters, and then the audio is processed according to the audio parameters.

[0110] In summary, the embodiments of the present disclosure provide an audio processing method, which first acquires user voice, recognizes the user voice to obtain the corresponding text content, and then converts the user voice in natural language form into target audio terms through a term conversion model, and the term conversion model is pre-trained based on natural voice text and sample annotation data, wherein the sample annotation data is audio terms annotated on the basis of natural voice text; then, based on the mapping relationship between preset audio parameters and audio terms, the target audio parameters corresponding to the target audio terms are queried to adjust the original audio according to the target audio adjustment parameters, so that ordinary users can use natural voice to control sound effect adjustment and realize personalized sound effect adjustment; the voice control operation is simple, and the user's natural voice is converted into audio terms that the machine can understand, and accurately adjusted to the sound effects expected by the user, which can meet the diversified needs of users and enhance the user's auditory experience.

[0111] like Figure 2 As shown, Figure 2 A schematic diagram of the structure of an audio processing device is provided for an embodiment of the present disclosure, the device comprising:

[0112] The acquisition module 201 is used to acquire user voice and identify text content corresponding to the user voice;

[0113] A term conversion module 202 is used to input the text content into a term conversion model to obtain a target audio term corresponding to the text content output by the term conversion model, wherein the term conversion model is trained based on natural speech text and sample annotation data, and the sample annotation data is audio terms annotated based on the natural speech text;

[0114] A query module 203, configured to query a target audio parameter corresponding to the audio term according to a preset mapping relationship between the audio parameter and the audio term;

[0115] The processing module 204 is used to process the original audio according to the target audio parameter.

[0116] As any optional implementation of the embodiments of the present disclosure, the term conversion module 202 is further used to train the term conversion model, including: obtaining natural speech text, and annotating audio terms based on the natural speech text to obtain the sample annotation data;

[0117] Inputting the natural speech text into an initial model to obtain a predicted audio term output by the initial model;

[0118] Calculating a loss function value according to the sample annotation data, the predicted audio term and a preset loss function;

[0119] If the loss function value is greater than the loss threshold, the model parameters of the initial model are adjusted until the loss function value is less than or equal to the loss threshold, thereby obtaining a converged term conversion model.

[0120] As any optional implementation of the embodiments of the present disclosure, the audio terms include: at least one of instrument frequency band terms, note frequency band terms, part-of-speech gain terms and sound position information.

[0121] As any optional implementation of the embodiments of the present disclosure, the target audio terms include: instrument frequency band terms, part-of-speech gain terms and sound position information;

[0122] The audio parameters include: the frequency range corresponding to the instrument frequency band term, the gain value corresponding to the part-of-speech gain term and the sound identification corresponding to the sound position information;

[0123] The processing module 204 is specifically used to: adjust the audio segment involved in the frequency range corresponding to the instrument frequency band term in the original audio according to the gain value corresponding to the part-of-speech gain term to obtain the target audio; and send the target audio to the speaker indicated by the speaker identifier.

[0124] As any optional implementation of the embodiments of the present disclosure, the term conversion module 202 is specifically used to extract target keywords from the text content, wherein the target keywords include at least one of musical instrument keywords, note keywords, part of speech keywords, and sound location keywords;

[0125] The target keyword is input into the term conversion model to obtain a target audio term corresponding to the target keyword output by the term conversion model.

[0126] As any optional implementation manner in the embodiments of the present disclosure, the processing module 204, after processing the original video according to the target audio parameter, is further configured to:

[0127] Sending the processed target audio to the corresponding speaker so that the speaker plays the target audio;

[0128] If a new user voice is acquired within the preset time period, extracting a first keyword from the new user voice;

[0129] When the first keyword matches the target keyword, processing the target audio according to the target audio parameter to obtain a first audio;

[0130] The first audio is sent to the speaker, so that the speaker plays the first audio.

[0131] As any optional implementation of the embodiments of the present disclosure, the acquisition module 201 is specifically used to pre-process the acquired user voice, and the pre-processing includes denoising and voice extraction;

[0132] The preprocessed user voice is recognized to obtain text content corresponding to the user voice.

[0133] In summary, the embodiments of the present disclosure provide an audio processing device, which first acquires user voice by an acquisition module, recognizes the user voice to obtain the corresponding text content, and then converts the user voice in natural language form into target audio terms through a term conversion model by a term conversion module, and the term conversion model is pre-trained based on natural voice text and sample annotation data, wherein the sample annotation data is audio terms annotated on the basis of natural voice text; then, the query module queries the target audio parameters corresponding to the target audio terms based on the mapping relationship between preset audio parameters and audio terms, so as to adjust the output audio according to the target audio parameters through the processing module, so that ordinary users can use natural voice to control sound effect adjustment and realize personalized sound effect adjustment; the voice control operation is simple, and the user's natural voice is converted into audio professional terms that can be understood by the machine, and accurately adjusted to the sound effects expected by the user, which can meet the diversified needs of users and enhance the user's auditory experience.

[0134] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, and the electronic device includes: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301, and the computer program implements each process of the audio processing method in the above method embodiment when executed by the processor 301. And the same technical effect can be achieved, and it will not be repeated here to avoid repetition.

[0135] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the audio processing method in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0136] The computer readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0137] An embodiment of the present invention provides a computer program product, which stores a computer program. When the computer program is executed by a processor, the various processes of the audio processing method in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0138] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0139] In the present disclosure, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0140] In the present disclosure, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0141] In the present disclosure, computer-readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0142] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0143] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio processing method, characterized in that: include: Acquire user voice and identify text content corresponding to the user voice; Inputting the text content into a term conversion model to obtain a target audio term corresponding to the text content output by the term conversion model, wherein the term conversion model is trained based on natural speech text and sample annotated data, and the sample annotated data is audio terms annotated based on the natural speech text; According to a preset mapping relationship between audio parameters and audio terms, querying a target audio parameter corresponding to the target audio term; The original audio is processed according to the target audio parameter.

2. The method according to claim 1, characterized in that The training method of the term conversion model comprises: Acquire natural speech text, and annotate audio terms based on the natural speech text to obtain the sample annotated data; Inputting the natural speech text into an initial model to obtain a predicted audio term output by the initial model; Calculating a loss function value according to the sample annotation data, the predicted audio term and a preset loss function; If the loss function value is greater than the loss threshold, the model parameters of the initial model are adjusted until the loss function value is less than or equal to the loss threshold, thereby obtaining a converged term conversion model.

3. The method according to claim 1 or 2, characterized in that: The audio terms include at least one of instrument frequency band terms, note frequency band terms, part-of-speech gain terms and sound position information.

4. The method according to claim 1, characterized in that: The target audio terms include: instrument frequency band terms, part-of-speech gain terms and sound position information; The target audio parameters include: a frequency range corresponding to the instrument frequency band term, a gain value corresponding to the part-of-speech gain term, and a sound identifier corresponding to the sound position information; The processing of the original audio according to the target audio parameter includes: According to the gain value corresponding to the part-of-speech gain term, the audio segment involved in the frequency range corresponding to the instrument frequency band term in the original audio is adjusted to obtain the target audio; The target audio is sent to the speaker indicated by the speaker identifier.

5. The method according to claim 1 or 2, characterized in that: The step of inputting the text content into a term conversion model to obtain a target audio term corresponding to the text content output by the term conversion model includes: Extracting target keywords from the text content, the target keywords including at least one of musical instrument keywords, note keywords, part of speech keywords, and sound location keywords; The target keyword is input into the term conversion model to obtain a target audio term corresponding to the target keyword output by the term conversion model.

6. The method according to claim 5, characterized in that After the original audio is processed according to the target audio parameter, the method further includes: Sending the processed target audio to a corresponding speaker so that the speaker plays the target audio; If a new user voice is acquired within the preset time period, extracting a first keyword from the new user voice; When the first keyword matches the target keyword, processing the target audio according to the target audio parameter to obtain a first audio; The first audio is sent to the speaker so that the speaker plays the first audio.

7. The method according to claim 1, characterized in that The acquiring of user voice and identifying text content corresponding to the user voice includes: Preprocessing the acquired user voice, wherein the preprocessing includes denoising and voice extraction; The preprocessed user voice is recognized to obtain text content corresponding to the user voice.

8. An audio processing device, characterized in that: include: An acquisition module, used to acquire user voice and identify text content corresponding to the user voice; A term conversion module, used for inputting the text content into a term conversion model, and obtaining a target audio term corresponding to the text content output by the term conversion model, wherein the term conversion model is trained based on natural speech text and sample annotation data, and the sample annotation data is audio terms annotated on the basis of the natural speech text; A query module, used to query the target audio parameter corresponding to the target audio term according to a preset mapping relationship between the audio parameter and the audio term; The processing module is used to process the original audio according to the target audio parameter.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the audio processing method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the audio processing method according to any one of claims 1 to 7 is implemented.

11. A vehicle, characterized in that: include: A speaker, and the audio processing device as claimed in claim 8, or an electronic device as claimed in claim 9.

Images