Audio analysis model training and audio analysis method, device, medium and product

Abstract

The embodiment of the invention relates to the technical field of natural language processing, and discloses an audio analysis model training and audio analysis method and device, a medium and a product. The method comprises the following steps: training a required sample audio file according to an audio analysis model, and executing format conversion and preprocessing training tasks; controlling the audio analysis model to execute a text generation training task according to a processed sample audio file obtained by format conversion and preprocessing of the training task; according to text content generated by the text generation training task, controlling the audio analysis model to execute a sensitive word recognition and masking training task; and according to a format conversion and preprocessing training task, a text generation training task and a sensitive word identification and masking training task, performing parameter adjustment on the audio analysis model to obtain a convergent target audio analysis model. The technical problem of inaccurate model training data caused by a specific format or noise contained in a sample audio file can be at least solved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to an audio analysis model training and audio analysis method, device, medium and product. Background Art

[0002] Corpus cleaning is an important step in natural language processing. Corpus cleaning refers to preprocessing the collected corpus to ensure the quality and applicability of the data. For audio or video files, the corpus cleaning process requires converting the speech content in the audio or video file into text.

[0003] However, the inventors found that there are at least the following technical problems in the related technology: If the existing audio-to-text technology is in the case of complex background noise, multiple speakers or non-standard Mandarin, recognition errors or incompleteness often occur, resulting in inaccurate generated text; these problems seriously affect the readability of the text and the accuracy of subsequent processing. If the existing audio-to-text technology is in the case of pure music background and complete silence, recognition errors or misjudgments often occur, resulting in inaccurate generated text.

[0004] Therefore, how to improve the accuracy of text generation is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] One purpose of the present application is to provide an audio parsing model training and an audio parsing method, at least to solve the technical problem of inaccurate model training data due to a specific format or because the sample audio file contains noise.

[0006] To achieve the above objectives, some embodiments of the present application provide the following aspects:

[0007] In a first aspect, some embodiments of the present application further provide an audio parsing model training method, comprising:

[0008] Perform format conversion and preprocessing training tasks based on the sample audio files required for audio parsing model training;

[0009] Controlling the audio parsing model to perform the text generation training task based on the processed sample audio file obtained by the format conversion and preprocessing training task;

[0010] Based on the text content generated by the text generation training task, control the audio parsing model to perform sensitive word recognition and masking training tasks;

[0011] According to the format conversion and preprocessing training tasks, the text generation training tasks, and the sensitive word recognition and masking training tasks, the parameters of the audio parsing model are adjusted to obtain a converged target audio parsing model.

[0012] In a second aspect, some embodiments of the present application further provide an audio parsing method, including:

[0013] Determine an audio file to be parsed; the audio file to be parsed includes at least one audio file obtained from the video file to be parsed;

[0014] The audio file to be parsed is input into the trained target audio parsing model, the text content corresponding to the audio file to be parsed is obtained, and the sensitive words in the text content are masked.

[0015] In a third aspect, some embodiments of the present application further provide an electronic device, comprising: one or more processors; and a memory storing computer program instructions, wherein the computer program instructions, when executed, cause the processor to perform the steps of the method described above.

[0016] In a fourth aspect, some embodiments of the present application further provide a computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method as described above.

[0017] In a fifth aspect, some embodiments of the present application further provide a computer program product, comprising a computer program / instruction, which implements the steps of the method described above when executed by a processor.

[0018] Compared with the related art, the solution provided by the embodiment of the present application can obtain sample audio files in a specific format and after preprocessing for model training by introducing format conversion and preprocessing training tasks, thereby avoiding inaccurate model training data due to specific formats or noise in sample audio files; by providing accurate sample audio file parameter model training, the conversion accuracy of the audio parsing model to the text content is guaranteed, thereby improving the accuracy of text conversion. By introducing sensitive word recognition and masking training tasks, sensitive words in the converted text content are recognized and converted, thereby improving the health and compliance of the audio parsing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0020] Figure 1 An exemplary flow chart of an audio parsing model training method provided according to some embodiments of the present application;

[0021] Figure 2An exemplary flow chart of another audio parsing model training method provided according to some embodiments of the present application;

[0022] Figure 3 An exemplary flow chart of an audio parsing model training method for a special field provided according to some embodiments of the present application;

[0023] Figure 4 An exemplary flow chart of another audio parsing model training method provided according to some embodiments of the present application;

[0024] Figure 5 An exemplary flow chart of an audio parsing method provided according to some embodiments of the present application;

[0025] Figure 6 This is an exemplary structural diagram of an electronic device provided according to some embodiments of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0028] First embodiment

[0029] The first embodiment of the present application relates to an audio parsing model training method. This embodiment can be applied to audio parsing based on corpus cleaning, such as Figure 1 As shown, the method may include the following steps:

[0030] S110, performing format conversion and preprocessing training tasks according to the sample audio files required for audio analysis model training.

[0031] The audio parsing model may refer to parsing the audio file to be parsed and converting the audio into text content. First, the audio file to be parsed is obtained, and according to the sample audio file required for the audio parsing model training, the format conversion and pre-processing training tasks are performed on the audio file to be parsed.

[0032] The format conversion training task may refer to converting the audio file to be parsed into a standard format required by the audio parsing model, for example, WAV or MP3.

[0033] The preprocessing training task may refer to preprocessing the audio file to be parsed to improve the accuracy of subsequent model training; the preprocessing includes but is not limited to noise reduction processing and gain adjustment.

[0034] Optionally, if the acquired file to be parsed is a video file, the video file needs to be processed, an audio file needs to be extracted from the video file, and then the audio file needs to be format converted and preprocessed.

[0035] Among them, the format conversion and preprocessing training tasks can realize the format conversion of sample audio files with different format requirements, and batch obtain sample audio files after format conversion that meet the format requirements; preprocess the sample audio files after format conversion to improve the accuracy of subsequent processing and provide training data guarantee for subsequent audio analysis model training tasks.

[0036] S120, controlling the audio analysis model to execute the text generation training task according to the processed sample audio file obtained by the format conversion and preprocessing training task.

[0037] After the format conversion and preprocessing of the audio file to be analyzed, a processed sample audio file is obtained, and the processed sample audio file is input into the audio analysis model to generate text.

[0038] S130. Based on the text content generated by the text generation training task, control the audio analysis model to perform sensitive word recognition and masking training tasks.

[0039] Sensitive words can refer to words with unhealthy connotations or uncivilized words. With the development of the Internet, more and more audio and video files and text content are transmitted through the Internet. It is inevitable that these audio and video files and text content will intentionally use sensitive words to destroy the network environment. In order to ensure the harmony and health of the network environment, it is necessary to identify and cover up sensitive words in the text content before publishing it.

[0040] The sensitive word recognition and masking training task refers to performing a sensitive word recognition and masking training task on the text content generated according to the text generation training task, so as to recognize and mask the sensitive words in the generated text content.

[0041] The sensitive word identification and masking training task can identify and mask sensitive words in the converted text content, so that the converted text content meets security and compliance, thereby improving the harmony and health of the audio analysis environment.

[0042] S140, adjusting parameters of the audio analysis model according to the format conversion and preprocessing training task, the text generation training task, and the sensitive word recognition and masking training task to obtain a converged target audio analysis model.

[0043] Among them, according to the format conversion and preprocessing training tasks, text generation training tasks, and sensitive word recognition and masking training tasks, the parameters of the audio parsing model are adjusted to fine-tune and optimize the audio parsing model to obtain a converged target audio parsing model to improve the accuracy of text conversion.

[0044] It is not difficult to find that compared with the related art, the solution provided in the embodiment of the present application can obtain a sample audio file in a specific format and after preprocessing for model training by introducing format conversion and preprocessing training tasks, thereby avoiding inaccurate model training data due to a specific format or noise in the sample audio file; by providing accurate sample audio file parameter model training, the conversion accuracy of the audio parsing model to the text content is guaranteed, and the accuracy of text conversion is improved. By introducing sensitive word recognition and masking training tasks to identify and convert sensitive words in the converted text content, the health and compliance of the audio parsing environment are improved.

[0045] Second embodiment

[0046] The second embodiment of the present application relates to an audio analysis model training method. The second implementation is a further optimization based on the first embodiment. The embodiment of the present invention can be combined with each optional solution in one or more of the above embodiments. Figure 2 As shown, the audio parsing model training method provided in the embodiment of the present invention may include the following steps:

[0047] S210. According to the first sample audio file required for the audio parsing model training, a format conversion training task is performed on the audio file to be parsed from the first format to the second format to obtain a second sample audio file.

[0048] The first sample audio file may refer to a processed sample file, that is, a sample audio file after format conversion and pre-processing.

[0049] In the embodiment of the present invention, when performing the audio file model training task, the sample audio file required first needs to convert the format of the audio file to be parsed to obtain a second sample audio file. The format conversion training task refers to performing a format conversion training task on the audio file to be parsed from a first format to a second format. The starting audio file format of the sample audio file is the first format; the second format is an audio file format that can be applied when the audio parsing model is trained so that the audio parsing model performs audio parsing.

[0050] S220. According to the first sample audio file required for the audio analysis model training, the second sample audio file after format conversion is subjected to a preprocessing training task to obtain a processed sample audio file.

[0051] The first sample audio file refers to a sample audio file after format conversion and preprocessing. In step S210, a second sample audio file is obtained after the format conversion training task is performed on the audio file to be parsed; in this step, a preprocessing training task is performed on the second sample audio file to obtain a processed sample audio file. At this time, the processed sample audio file is a sample audio file after format conversion and preprocessing, that is, the first sample audio file. The preprocessing training task includes but is not limited to noise reduction and gain adjustment training tasks.

[0052] Among them, the embodiment of the present invention performs format conversion and preprocessing training tasks on the audio files to be parsed, which can greatly improve the efficiency of corpus cleaning, reduce manual intervention, and improve the degree of automation of the overall workflow, and is suitable for large-scale data processing scenarios.

[0053] S230, controlling the audio analysis model to execute the text generation training task according to the processed sample audio file obtained by the format conversion and preprocessing training task.

[0054] Among them, after determining the processed sample audio file, that is, after the corpus of the audio file to be parsed is cleaned, the audio parsing model is controlled to perform the text generation training task.

[0055] As an optional but non-limiting implementation, the processed sample audio file obtained according to the format conversion and preprocessing training task controls the audio parsing model to perform the text generation training task, including but not limited to steps A1-A2:

[0056] Step A1: Input the processed sample audio file obtained according to the format conversion and preprocessing training tasks into the audio analysis model.

[0057] Step A2: Use the audio parsing module to perform a text generation training task.

[0058] The audio parsing model is used to perform a text generation training task on the processed sample audio file, so as to identify and output text for the audio file to be parsed.

[0059] S240. Based on the text content generated by the text generation training task, control the audio parsing model to perform sensitive word recognition and masking training tasks.

[0060] After executing the text generation training task, the generated text content contains sensitive words. If the text content is disseminated, the appearance of sensitive words will affect the environment in which the text content is disseminated. Therefore, it is necessary to identify and mask the sensitive words in the text content. In the embodiment of the present invention, the audio parsing model is controlled to perform the sensitive word identification and masking training task to identify and mask the sensitive words in the text content.

[0061] As an optional but non-limiting implementation, the text content generated according to the text generation training task controls the audio parsing model to perform the sensitive word recognition and masking training task, including but not limited to steps B1-B3:

[0062] Step B1: Build a sensitive vocabulary library.

[0063] Step B2: comparing the text content generated by the text generation training task with the sensitive words in the sensitive vocabulary library word by word to determine whether there are sensitive words in the generated text content.

[0064] Step B3: If there are sensitive words in the generated text content, the sensitive words are masked using a preset masking method.

[0065] Wherein, a sensitive vocabulary library is constructed, and the sensitive vocabulary library may refer to a dynamically updated sensitive vocabulary library. After a new sensitive word appears, the new sensitive word is added to the sensitive vocabulary library.

[0066] In the embodiment of the present invention, controlling the audio parsing model to perform sensitive word identification and masking training tasks may refer to comparing the generated text content with the sensitive words in the sensitive vocabulary library word by word, automatically identifying the sensitive words in the text content and masking them. The masking may refer to masking the sensitive words according to a preset masking method. In the embodiment of the present invention, the appearing sensitive words can be automatically identified and marked, thereby improving the security and compliance of the corpus cleaning process.

[0067] Optionally, the sensitive word identification and masking training task can be executed synchronously with the text generation training task, and sensitive words are identified and masked on the text content when the text content is generated, so as to improve the efficiency of text generation.

[0068] In an optional scheme of an embodiment of the present invention, when controlling the audio analysis model to perform sensitive word recognition and masking training tasks, the audio analysis model is also required to perform context analysis training tasks to analyze the context of the location where sensitive words appear when sensitive words appear, determine the specific method of masking sensitive words, reduce false alarms and missed alarms, and ensure the accuracy of sensitive word recognition.

[0069] S250, adjusting parameters of the audio analysis model according to the format conversion and preprocessing training task, the text generation training task, and the sensitive word recognition and masking training task to obtain a converged target audio analysis model.

[0070] After completing each training task, the audio parsing model is parameterized to obtain a converged target audio parsing model. By fine-tuning and optimizing the audio parsing model, the accuracy of text conversion is effectively improved. In particular, in environments with high noise, multiple speakers, and non-standard Mandarin, the present invention can significantly improve the conversion effect.

[0071] As an optional but non-limiting implementation, the audio parsing model is parameter-adjusted according to the format conversion and preprocessing training task, the text generation training task, and the sensitive word recognition and masking training task to obtain a converged target audio parsing model, including but not limited to steps C1-C3:

[0072] Step C1: Determine the first loss function value of the format conversion and preprocessing training task, the second loss function value of the text generation training task, and the third loss function value of the sensitive word recognition and masking training task.

[0073] Step C2: The first loss function value, the second loss function value and the third loss function value are fused according to the loss weights of their respective training tasks.

[0074] Step C3: adjusting the parameters of the audio analysis model according to the loss function value obtained by the loss fusion until the audio analysis model converges to obtain the target audio analysis model.

[0075] Among them, when controlling the audio parsing model to perform the text generation training task, it is necessary not only to determine the first loss function value of the format conversion and preprocessing training task, but also to determine the second loss function value of the text generation training task and the third loss function value of the sensitive word recognition and masking training task. The first loss function value, the second loss function value and the third loss function value are fused according to the loss weights of their respective training tasks, so that the audio parsing model can learn the format conversion and preprocessing information of the sample audio file, as well as the sensitive word recognition and masking information while performing text generation training, so that the audio parsing model can be combined with multiple dimensions such as format conversion and preprocessing information, sensitive word recognition and masking information, and text generation during training to realize the training update of the audio parsing model, and further improve the text conversion precision, accuracy and compliance of the audio parsing model for the audio files to be parsed.

[0076] It is not difficult to find that in the embodiment of the present application, by performing format conversion and preprocessing training tasks on the audio files to be parsed, the efficiency of corpus cleaning can be greatly improved, manual intervention is reduced, and the automation level of the overall workflow is improved, which is suitable for large-scale data processing scenarios. Automatic identification and marking of sensitive words improves the security and compliance of the corpus cleaning process. Analyze the context of the location where sensitive words appear, determine the specific method of masking sensitive words, reduce false positives and false negatives, and ensure the accuracy of sensitive word recognition. Combine format conversion and preprocessing information, sensitive word recognition and masking information, and text generation and other dimensions to realize the training and update of the audio parsing model, and further improve the text conversion precision, accuracy and compliance of the audio parsing model for the audio files to be parsed.

[0077] Third embodiment

[0078] The third embodiment of the present application relates to an audio analysis model training method. The second implementation is a further optimization based on the first embodiment. The embodiment of the present invention can be combined with each optional solution in one or more of the above embodiments. Figure 3 As shown, the audio parsing model training method provided in the embodiment of the present invention may include the following steps S310-S320:

[0079] S310: Obtain sample audio files in special fields, perform format conversion and preprocessing training tasks, text generation training tasks, and sensitive word recognition and masking training tasks, and determine audio features of sample audio files in special fields.

[0080] S320: Adjusting parameters of the audio analysis model according to the audio features to obtain a converged target audio analysis model.

[0081] Among them, an embodiment of the present invention provides an audio analysis model training method for special fields, and the special fields may refer to education and training, news media, content review, and intelligent customer service. The model training method is consistent with the model training method of embodiment one or embodiment two, and both need to perform format conversion and preprocessing training tasks, text generation training tasks, and sensitive word recognition and masking training tasks on the audio files to be analyzed, and adjust parameters with the audio analysis model to obtain a converged target audio analysis model.

[0082] Since audio files in special fields contain a large number of professional terms, in an embodiment of the present invention, when performing a training task on an audio file to be parsed in a special field, it is necessary to determine the audio features of the sample audio files in the special field, and adjust the parameters of the audio parsing model based on the audio features to obtain a converged target audio parsing model.

[0083] It is not difficult to find that in the embodiments of the present application, by constructing a training method for an audio parsing model in a special field, it can be widely used in multiple fields such as news media, education and training, content review, and intelligent customer service, with significant social and economic benefits. Scientific analysis and experimental results have proved that the present invention performs well in multiple practical application scenarios and has broad application prospects.

[0084] Fourth embodiment

[0085] In an optional solution of the embodiment of the present invention, see Figure 4 In the inference process of the audio parsing model, the underlying principle is to first pass the audio signal through a pre-processing module for feature extraction. This module usually converts the audio signal into a Mel-spectrogram, because the spectrogram can well represent the time-frequency characteristics of the audio. Next, the Mel-spectrogram is input into the Transformer encoder, which captures the long-range dependencies between audio features through the self-attention mechanism. The decoder is responsible for mapping these features into text sequences in the target language. In this process, the decoder also uses the previously predicted text information to make the next prediction, thereby gradually generating accurate text output in the speech stream. The inference of the audio parsing model relies on the powerful language model capabilities obtained through large-scale training, so that it can accurately interpret the speech content in different contexts.

[0086] In order to fine-tune or customize the audio parsing model, it is usually necessary to retrain it on data in a specific domain or language. The steps of fine-tuning usually include the following aspects. First, collect and prepare a speech dataset related to the target task, which should cover the language, accent or specific domain that the model needs to handle. Then, use this data to fine-tune the pre-trained model, usually training at a lower learning rate to prevent the model from overfitting. During the customization process, you can also further optimize the quality of the output by adjusting the model's decoding parameters, such as setting different beam search widths or adjusting the weights of the language model. If you need to improve performance in a specific field, you can also add custom vocabularies and grammatical constraints to make the model more suitable for the recognition of certain proper nouns or fixed expressions.

[0087] Optional, Figure 4 In the embodiment of the present invention, a variety of audio data are used for training, among which English transcription represents direct English speech transcription; Any-to-English speech transcription represents non-English speech, translated into English text; Non-English transcription represents direct transcription of non-English speech into text of the corresponding language, such as Chinese into Chinese. No speech represents background music or no audio.

[0088] Among them, the Multi-task Training Format is used for different tasks of the model. LANGUAGE TAG represents the language of the input speech; NO SPEECH represents the input without a language label; TRANSCRIBED represents the transcription label; TRANSLATE represents the translation label for the model. Time-aligned transcription represents the time-aligned transcription, text-aligned speech; Text-only transcription represents direct transcription and allows the model to be fine-tuned (based on a specific dataset).

[0089] In sequence-to-sequence learning, 2xconv1 D+GELU refers to the use of a 2-layer dimensional product and activation function (Gaussian Error Line Unit, GELU) to process the input audio. The blue part is the encoder (Encoder Blocks), which uses the Transformer encoder block, including the multi-layer perceptron (MLP) self-attention mechanism and sine position, responsible for extracting information from the input features. The red part is the decoder (Decoder Blocks), which also uses MLP and cross-attention mechanism. The task of the decoder is to generate the corresponding text.

[0090] It is not difficult to find that in the embodiments of the present application, the audio analysis model is trained by a large number of data sets, which can accurately interpret the speech content in different contexts and effectively improve the accuracy of text conversion.

[0091] Fifth embodiment

[0092] The fifth embodiment of the present application relates to an audio parsing method. The fifth implementation is an improvement based on the first embodiment, the second embodiment and the third embodiment. The specific improvement is that the target audio-to-text model obtained by the training method of the audio-to-text model generates text for the audio file to be parsed. Figure 5 As shown, the audio analysis method includes steps S510-S520:

[0093] S510: Determine an audio file to be parsed; the audio file to be parsed includes at least one audio file obtained from the video file to be parsed.

[0094] S520: Input the audio file to be parsed into the trained target audio parsing model, obtain the text content corresponding to the audio file to be parsed, and mask the sensitive words in the text content.

[0095] The target audio parsing model is obtained by using any of the audio parsing model training methods described in the above embodiments. After entering the model application stage, the audio file to be parsed can be input into the trained target audio parsing model to output the text content corresponding to the audio file to be parsed.

[0096] The embodiment of the present invention provides an audio parsing method, which determines an audio file to be parsed, inputs the audio file to be parsed into a trained target audio parsing model, and obtains the text content corresponding to the audio file to be parsed. The technical solution of the embodiment of the present invention effectively improves the accuracy of text conversion, reduces manual intervention, and improves the automation of the overall workflow, which is suitable for large-scale data processing scenarios.

[0097] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0098] Sixth embodiment

[0099] In addition, some embodiments of the present application also provide an electronic device. The electronic device may be a digital computer in various forms, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, etc. The electronic device may also be a mobile device in various forms, such as a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices.

[0100] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the steps of the method provided in any one or more of the above embodiments. Figure 6 An exemplary structural diagram of the electronic device is disclosed. Figure 6 As shown, the electronic device includes: one or more processors 1101, memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Among them, the components shown in this article, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0101] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0102] The input device 1103 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 1104 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0103] To provide interaction with a user, the electronic device may be a computer. The computer has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0104] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium, and when the computer program / instruction is executed by a processor, the steps of the method provided by any one or more of the above embodiments are implemented. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more computer-readable instructions.

[0105] The memory 1102 can be used as a non-transient computer-readable storage medium, which can be used to store non-transient software programs, non-transient computer executable programs and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 1102, so as to implement the program instructions / modules corresponding to the method provided by any one or more embodiments in the embodiments of the present application.

[0106] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely arranged relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0107] It should be noted that the computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0108] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0109] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).

[0110] In the above-described embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. For example, it can be implemented by using an application specific integrated circuit (ASIC), a general-purpose computer or any other similar hardware device. In certain embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented by hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0111] The computer program product provided in the embodiment of the present application includes one or more computer programs / instructions, and when the computer program / instructions are executed by the processor, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive so lid state di sk (SSD)), etc.

[0112] The flow chart or block diagram in the accompanying drawings shows the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim may also be implemented by one unit or device through software or hardware. The words "first", "second", etc. are only used to distinguish the description, and do not indicate any particular order, nor can they be understood as indicating or implying relative importance.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily mention changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims, and the above embodiments should be regarded as exemplary and non-restrictive.

Claims

1. A method for training an audio analysis model, characterized in that: The method comprises: Perform format conversion and preprocessing training tasks based on the sample audio files required for audio parsing model training; Controlling the audio parsing model to perform the text generation training task based on the processed sample audio file obtained by the format conversion and preprocessing training task; Based on the text content generated by the text generation training task, control the audio parsing model to perform sensitive word recognition and masking training tasks; According to the format conversion and preprocessing training tasks, the text generation training tasks, and the sensitive word recognition and masking training tasks, the parameters of the audio parsing model are adjusted to obtain a converged target audio parsing model.

2. The method according to claim 1, characterized in that The sample audio files required for audio analysis model training are used to perform format conversion and preprocessing training tasks, including: According to the first sample audio file required for the audio parsing model training, a format conversion training task is performed on the audio file to be parsed from the first format to the second format to obtain a second sample audio file; the starting audio file format of the sample audio file is the first format; the second format is an audio file format applicable to the audio parsing model when training the audio parsing model so that the audio parsing model performs audio parsing; According to the first sample audio file required for the audio analysis model training, the second sample audio file after format conversion is subjected to a preprocessing training task to obtain a processed sample audio file; the preprocessing training task includes noise reduction and gain adjustment training tasks.

3. The method according to claim 1, characterized in that The processed sample audio file obtained according to the format conversion and preprocessing training task controls the audio parsing model to perform the text generation training task, including: Inputting the processed sample audio file obtained according to the format conversion and preprocessing training tasks into the audio parsing model; The audio parsing module is used to perform text generation training tasks.

4. The method according to claim 1, characterized in that: The text content generated by the text generation training task controls the audio parsing model to perform the sensitive word recognition and masking training task, including: Build a sensitive vocabulary database; Compare the text content generated by the text generation training task with the sensitive words in the sensitive vocabulary library word by word to determine whether there are sensitive words in the generated text content; If there are sensitive words in the generated text content, the sensitive words are masked using a preset masking method.

5. The method according to claim 1, characterized in that The audio parsing model is parameterized according to the format conversion and preprocessing training task, the text generation training task, and the sensitive word recognition and masking training task to obtain a converged target audio parsing model, including: Determine a first loss function value for a format conversion and preprocessing training task, a second loss function value for a text generation training task, and a third loss function value for a sensitive word identification and masking training task; The first loss function value, the second loss function value and the third loss function value are subjected to loss fusion according to the loss weights of the respective training tasks; The parameters of the audio analysis model are adjusted according to the loss function value obtained by the loss fusion until the audio analysis model converges to obtain the target audio analysis model.

6. The method according to claim 1, characterized in that The method further comprises: Obtain sample audio files in special fields, perform format conversion and preprocessing training tasks, text generation training tasks, and sensitive word recognition and masking training tasks, and determine the audio features of sample audio files in special fields; Parameters of the audio analysis model are adjusted according to the audio features to obtain a converged target audio analysis model.

7. An audio analysis method, characterized in that: The target audio analysis model is obtained by using the audio analysis model training method described in any one of claims 1 to 6, wherein the audio analysis method comprises: Determine an audio file to be parsed; the audio file to be parsed includes at least one audio file obtained from the video file to be parsed; The audio file to be parsed is input into the trained target audio parsing model, the text content corresponding to the audio file to be parsed is obtained, and the sensitive words in the text content are masked.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the audio parsing model training method as described in any one of claims 1 to 6 or the audio parsing method as described in claim 7.

9. A computer readable medium having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the audio analysis model training method described in any one of claims 1 to 6 or the audio analysis method described in claim 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the audio analysis model training method described in any one of claims 1 to 6 or the audio analysis method described in claim 7 are implemented.

Images