Audio recognition method, device, electronic device and readable medium based on multiple models

Abstract

The embodiments of the present disclosure disclose a multi-model-based audio recognition method, device, electronic device and readable medium. A specific implementation of the method includes: performing orderly segmentation processing on the target audio to generate a segmented audio sequence; inputting the segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group; selecting at least one segmented audio recognition result from the segmented audio recognition result group as a target audio recognition candidate result group; in response to determining that the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1, determining the audio recognition model corresponding to the target audio recognition candidate result group as the target audio recognition model; inputting the remaining segmented audio of the target audio into the target audio recognition model to obtain the target audio recognition result. This implementation improves the accuracy of audio recognition and the efficiency of audio recognition.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computers, and in particular to a method, device, electronic device, and readable medium for audio recognition based on multiple models. Background Art

[0002] Currently, the common method for audio recognition is that the user or the system selects one or more audio recognition models to recognize the audio provided by the user based on some predefined parameters, and after the model recognition is completed, one or more recognition results are presented for the user to choose.

[0003] However, the above method usually has the following technical problems:

[0004] First, selecting one or more audio recognition models based on some predefined parameters requires the user to understand the meaning of the parameters. When the user is not familiar with the meaning of the parameters, it is usually impossible to accurately select the audio recognition model, resulting in a low accuracy rate of audio recognition;

[0005] Second, after selecting an audio recognition model, the user cannot perceive the audio recognition results in real time. When multiple models are used for recognition, too many computing resources will be occupied, and the user will need to spend a long time to select the recognition result. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a multi-model-based audio recognition method, apparatus, electronic device, and computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide an audio recognition method based on multiple models, the method comprising: in response to detecting an audio recognition selection operation acting on a target audio, performing an orderly segmentation process on the target audio to generate a segmented audio sequence; for each segmented audio in the segmented audio sequence, performing the following processing steps: inputting the segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group, wherein the audio recognition model in the audio recognition model group corresponds to the segmented audio recognition result in the segmented audio recognition result group, and each audio recognition model in the audio recognition model group is different; According to the various selection information corresponding to the above-mentioned segmented audio recognition result group submitted by the user, at least one segmented audio recognition result is selected from the above-mentioned segmented audio recognition result group as the target audio recognition candidate result group; it is determined whether the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1; in response to determining that the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1, the audio recognition model corresponding to the above-mentioned target audio recognition candidate result group is determined as the target audio recognition model; the remaining segmented audio of the above-mentioned target audio is input into the above-mentioned target audio recognition model to obtain the target audio recognition result.

[0009] In a second aspect, some embodiments of the present disclosure provide an audio recognition device based on multiple models, the device comprising: a segmentation unit, configured to, in response to detecting an audio recognition selection operation acting on a target audio, perform orderly segmentation processing on the target audio to generate a segmented audio sequence; a segmented audio recognition unit, configured to, for each segmented audio in the segmented audio sequence, perform the following processing steps: input the segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group, wherein the audio recognition models in the audio recognition model group correspond to the segmented audio recognition results in the segmented audio recognition result group, and each audio recognition model in the audio recognition model group corresponds to the segmented audio recognition result in the segmented audio recognition result group. difference; according to the various selection information corresponding to the above-mentioned segmented audio recognition result group submitted by the user, select at least one segmented audio recognition result from the above-mentioned segmented audio recognition result group as the target audio recognition candidate result group; determine whether the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1; the determination unit is configured to, in response to determining that the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1, determine the audio recognition model corresponding to the above-mentioned target audio recognition candidate result group as the target audio recognition model; the input unit is configured to input the remaining segmented audio of the above-mentioned target audio into the above-mentioned target audio recognition model to obtain the target audio recognition result.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the multi-model-based audio recognition method of some embodiments of the present disclosure, the accuracy of audio recognition is improved, and the recognition efficiency of audio is improved. Specifically, the reason for the low accuracy of audio recognition is that: according to some pre-defined parameters, one or more audio recognition models are selected, and the user needs to understand the meaning of the parameters. When the user is not familiar with the meaning of the parameters, the audio recognition model cannot be accurately selected, resulting in a low accuracy of audio recognition. Based on this, the multi-model-based audio recognition method of some embodiments of the present disclosure, first, in response to detecting an audio recognition selection operation acting on the target audio, the above-mentioned target audio is orderly segmented to generate a segmented audio sequence. Thus, the audio can be gradually recognized by each model to select a suitable audio recognition model to avoid occupying too many computing resources. Then, for each segmented audio in the above segmented audio sequence, the following processing steps are performed: First, the above segmented audio is input into a pre-trained audio recognition model group to obtain a segmented audio recognition result group. Among them, the audio recognition model in the above-mentioned audio recognition model group corresponds to the segmented audio recognition result in the above-mentioned segmented audio recognition result group, and each audio recognition model in the above-mentioned audio recognition model group is different. Thus, data support can be provided for selecting a suitable audio recognition model. Then, according to each selection information corresponding to the above-mentioned segmented audio recognition result group submitted by the user, at least one segmented audio recognition result is selected from the above-mentioned segmented audio recognition result group as the target audio recognition candidate result group. Thus, it is convenient to select an audio recognition model with a higher accuracy rate of audio recognition according to the user's selection of the segmented audio recognition result. Then, it is determined whether the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1. Then, in response to determining that the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1, the audio recognition model corresponding to the above-mentioned target audio recognition candidate result group is determined as the target audio recognition model. Thus, according to the recognition results of multiple models for the same audio, the audio recognition model with the best recognition accuracy can be selected. At the same time, since the target audio is segmented in advance, it is avoided to occupy too many computing resources and recognition time. Finally, the remaining segmented audio of the target audio is input into the target audio recognition model to obtain the target audio recognition candidate result, thereby improving the accuracy of audio recognition and the efficiency of audio recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flowchart of some embodiments of a multi-model-based audio recognition method according to the present disclosure;

[0015] Figure 2 is a schematic diagram of the structure of some embodiments of the multi-model based audio recognition device according to the present disclosure;

[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 1 is a flow chart of some embodiments of the multi-model based audio recognition method according to the present disclosure. 100 of some embodiments of the multi-model based audio recognition method according to the present disclosure is shown. The multi-model based audio recognition method comprises the following steps:

[0024] Step 101, in response to detecting an audio recognition selection operation acting on a target audio, orderly segmentation processing is performed on the target audio to generate a segmented audio sequence.

[0025] In some embodiments, the execution subject (e.g., a computing device) of the multi-model based audio recognition method can, in response to detecting an audio recognition selection operation acting on the target audio, perform orderly segmentation processing on the target audio to generate a segmented audio sequence. Here, the target audio can be the audio to be recognized. For example, the target audio can be a recording of a person communicating uploaded by a user. The audio recognition selection operation can refer to an operation of selecting the target audio for audio recognition. For example, the audio recognition selection operation can be clicking on the target audio for audio recognition. Here, the duration of the target audio is an audio that is greater than or equal to a preset duration. For example, the preset duration can be 5 minutes. In practice, the execution subject can perform orderly segmentation processing on the target audio according to the target duration (it can be segmented according to a certain duration, for example, divided according to 5 seconds as a unit duration) to generate a segmented audio sequence. For example, the target duration can be 5 seconds, and the execution subject can perform orderly segmentation processing on the target audio in units of 5 seconds to generate a segmented audio sequence. For segmented audio of less than 5 seconds, segmentation is performed according to the actual audio duration. That is, the duration of the segmented audio is equal to or less than 5 seconds.

[0026] Optionally, before the target audio is sequentially segmented to generate a segmented audio sequence, the method further includes:

[0027] The first step is to receive the audio to be recognized submitted by the user. In practice, the audio to be recognized submitted by the user's corresponding user terminal can be received through a wired connection or a wireless connection. Here, the audio to be recognized can refer to the audio from which the modal particles have been removed.

[0028] The second step is to remove the audio segments that meet the blank audio condition from the audio to be recognized, so as to update the audio to be recognized, and use the updated audio to be recognized as the target audio. The blank audio condition is that the audio segment does not contain the target voice. Here, the target voice can refer to the voice of the core person in the audio. The core person can refer to the person who speaks multiple times in the audio.

[0029] In some optional implementations of some embodiments, the execution subject may further segment the target audio to generate segmented audio groups through the following steps:

[0030] The first step is to determine the duration of the target audio.

[0031] In the second step, in response to the duration being greater than or equal to the preset duration, the target audio is segmented according to the first preset segmentation duration to generate a segmented audio group. Here, the preset duration may be 1 hour. The first preset segmentation duration may be 10 seconds. In practice, the target audio may be segmented according to 10 seconds as a segmentation unit duration to generate a segmented audio group. That is, the duration of the segmented audio is 10 seconds or less.

[0032] In the third step, in response to the above-mentioned duration being less than the above-mentioned preset duration, the above-mentioned target audio is segmented according to the second preset segmentation duration to generate a segmented audio group. The above-mentioned second preset segmentation duration is less than the above-mentioned first preset segmentation duration. The second preset segmentation duration can be 1 minute. In practice, the above-mentioned target audio can be segmented according to 1 minute as a segmentation unit duration to generate a segmented audio group. That is, the duration of the segmented audio is 1 minute or less.

[0033] Step 102: for each segmented audio in the segmented audio group, perform the following processing steps:

[0034] Step 1021, input the above segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group.

[0035] In some embodiments, the execution subject may input the segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group. The audio recognition model in the audio recognition model group corresponds to the segmented audio recognition result in the segmented audio recognition result group, and each audio recognition model in the audio recognition model group is different. Here, the audio recognition model in the audio recognition model group may be a pre-trained neural network model for recognizing audio. The audio recognition model in the audio recognition model group may also be: an acoustic model, a language model. For example, the audio recognition model may also be: a Gaussian mixture model, an N-gram model, a Trie tree. Each audio recognition model in the audio recognition model group is an audio recognition model of a different type. For example, the type of audio recognition model may include but is not limited to: a model for recognizing conference audio, a model for recognizing audio of a user speaking and communicating, and a model for recognizing audio of a teacher lecturing. Here, the audio recognition model may be a neural network model trained with audio as input and recognized audio text as output. For example, the audio recognition model may be a pre-trained convolutional neural network model or a recurrent neural network model. The segmented audio recognition result may represent the recognized audio text of the segmented audio.

[0036] In practice, the execution entity may input the segmented audio into each audio recognition model in a pre-trained audio recognition model group to obtain a segmented audio recognition result group.

[0037] Step 1022: According to the selection information corresponding to the segmented audio recognition result group submitted by the user, at least one segmented audio recognition result is selected from the segmented audio recognition result group as the target audio recognition candidate result group.

[0038] In some embodiments, the execution entity may select at least one segmented audio recognition result from the segmented audio recognition result group as the target audio recognition candidate result group based on the selection information corresponding to the segmented audio recognition result group submitted by the user. The selection information is information submitted by the user for selecting the segmented audio recognition result in the segmented audio recognition result group. The segmented audio recognition results in the segmented audio recognition result group correspond to the selection information in the above-mentioned selection information. The selection information may indicate selecting the segmented audio recognition result corresponding to the above-mentioned selection information or not selecting the segmented audio recognition result corresponding to the above-mentioned selection information.

[0039] In practice, the execution subject may select the segmented audio recognition result represented by the corresponding selection information from the segmented audio recognition result group as the target audio recognition candidate result to obtain the target audio recognition candidate result group.

[0040] In some optional implementations of some embodiments, the above-mentioned execution entity can, for each of the above-mentioned selection information, in response to determining that the above-mentioned selection information represents the selection of the segmented audio recognition result corresponding to the above-mentioned selection information, determine the segmented audio recognition result corresponding to the above-mentioned selection information as the target audio recognition candidate result.

[0041] Optionally, before step 1022, the method further includes:

[0042] The first step is to control the associated audio playback device to play the above-mentioned segmented audio, so that the user can select the segmented audio recognition result in the above-mentioned segmented audio recognition result group according to the segmented audio played. Here, the associated audio playback device can refer to a sound playback device that is communicatively connected to the above-mentioned execution subject. For example, the audio playback device can be a speaker. Here, the selection operation can mean selecting a certain segmented audio recognition result or not selecting a certain segmented audio recognition result.

[0043] The second step is to receive the user's selection information for each segmented audio recognition result in the segmented audio recognition result group.

[0044] The relevant content in the optional part is used as an inventive point of the present disclosure, thereby solving the technical problem 2 mentioned in the background technology: "After selecting the audio recognition model, the user cannot perceive the audio recognition result in real time, when multiple models are used for recognition, too many computing resources will be occupied, and the user needs to spend a long time to select the recognition result.". The factors that occupy too many computing resources and the user needs to spend a long time to select the recognition result are often as follows: After selecting the audio recognition model, the user cannot perceive the audio recognition result in real time, when multiple models are used for recognition, too many computing resources will be occupied, and the user needs to spend a long time to select the recognition result. If the above factors are solved, the effect of reducing the waste of computing resources can be achieved. In order to achieve this effect, first, the associated audio playback device is controlled to play the above-mentioned segmented audio, so that the user can select the segmented audio recognition result in the above-mentioned segmented audio recognition result group according to the segmented audio played. As a result, when multiple models recognize the audio, the user can participate in it. That is, the user can compare the audio content he hears with the recognition result of the model for the audio, so as to select a model with a higher audio recognition rate. In addition, because the audio is segmented, it avoids taking up too many computing resources. Then, the user's selection information for each segmented audio recognition result in the segmented audio recognition result group is received. Thus, an audio recognition model with the best audio recognition rate can be selected according to the user's selection information, avoiding the user having to spend a long time selecting the recognition result.

[0045] Step 1023, determining whether the number of target audio recognition candidate results included in the above target audio recognition candidate result group is 1.

[0046] In some embodiments, the execution entity may determine whether the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1.

[0047] Optionally, in response to determining that the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is greater than 1, each audio recognition model corresponding to the above-mentioned target audio recognition candidate result group is used as an updated audio recognition model group, and the segmented audio group from which the above-mentioned segmented audio is removed is used as an updated segmented audio group, and the above-mentioned processing steps are performed again.

[0048] In some embodiments, the above-mentioned execution entity can, in response to determining that the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is greater than 1, use the respective audio recognition models corresponding to the above-mentioned target audio recognition candidate result group as the updated audio recognition model group, use the segmented audio group from which the above-mentioned segmented audio is removed as the updated segmented audio group, and execute the above-mentioned processing steps again.

[0049] Optionally, the execution subject may further perform the processing steps again through the following steps:

[0050] In the first step, the target execution number is increased by 1. The target execution number is initially 1, and the target execution number represents the execution number of the processing step.

[0051] The second step is to determine whether the target execution times are less than or equal to a preset times, wherein the preset times are: the rounded-up value of the target value, and the target value is: half of the number of segmented audios included in the segmented audio group.

[0052] In the third step, in response to determining that the target execution times is less than or equal to the preset times, the processing steps are executed again.

[0053] This can reduce the usage time of multiple models and the waste of computing resources.

[0054] Step 103: In response to determining that the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1, determine the audio recognition model corresponding to the target audio recognition candidate result group as the target audio recognition model.

[0055] In some embodiments, in response to determining that the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1, the execution entity may determine the audio recognition model corresponding to the target audio recognition candidate result group as the target audio recognition model.

[0056] Step 104: input the remaining segmented audio of the target audio into the target audio recognition model to obtain the target audio recognition result.

[0057] In some embodiments, the execution subject may input the remaining segmented audio of the target audio into the target audio recognition model to obtain the target audio recognition result. Here, the remaining segmented audio of the target audio may refer to each segmented audio in the segmented audio sequence that is not input into the target audio recognition model.

[0058] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the multi-model-based audio recognition method of some embodiments of the present disclosure, the accuracy of audio recognition is improved, and the recognition efficiency of audio is improved. Specifically, the reason for the low accuracy of audio recognition is that: according to some pre-defined parameters, one or more audio recognition models are selected, and the user needs to understand the meaning of the parameters. When the user is not familiar with the meaning of the parameters, the audio recognition model cannot be accurately selected, resulting in a low accuracy of audio recognition. Based on this, the multi-model-based audio recognition method of some embodiments of the present disclosure, first, in response to detecting an audio recognition selection operation acting on the target audio, the above-mentioned target audio is orderly segmented to generate a segmented audio sequence. Thus, the audio can be gradually recognized by each model to select a suitable audio recognition model to avoid occupying too many computing resources. Then, for each segmented audio in the above segmented audio sequence, the following processing steps are performed: First, the above segmented audio is input into a pre-trained audio recognition model group to obtain a segmented audio recognition result group. Among them, the audio recognition model in the above-mentioned audio recognition model group corresponds to the segmented audio recognition result in the above-mentioned segmented audio recognition result group, and each audio recognition model in the above-mentioned audio recognition model group is different. Thus, data support can be provided for selecting a suitable audio recognition model. Then, according to each selection information corresponding to the above-mentioned segmented audio recognition result group submitted by the user, at least one segmented audio recognition result is selected from the above-mentioned segmented audio recognition result group as the target audio recognition candidate result group. Thus, it is convenient to select an audio recognition model with a higher accuracy rate of audio recognition according to the user's selection of the segmented audio recognition result. Then, it is determined whether the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1. Then, in response to determining that the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1, the audio recognition model corresponding to the above-mentioned target audio recognition candidate result group is determined as the target audio recognition model. Thus, according to the recognition results of multiple models for the same audio, the audio recognition model with the best recognition accuracy can be selected. At the same time, since the target audio is segmented in advance, it is avoided to occupy too many computing resources and recognition time. Finally, the remaining segmented audio of the target audio is input into the target audio recognition model to obtain the target audio recognition result, thereby improving the accuracy of audio recognition and the efficiency of audio recognition.

[0059] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an audio recognition device based on a multi-model. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0060] like Figure 2 As shown, some embodiments of the multi-model-based audio recognition device 200 include: a segmentation unit 201, a segmented audio recognition unit 202, a determination unit 203 and an input unit 204. The segmentation unit 201 is configured to, in response to detecting an audio recognition selection operation acting on the target audio, perform orderly segmentation processing on the target audio to generate a segmented audio sequence; the segmented audio recognition unit 202 is configured to perform the following processing steps for each segmented audio in the segmented audio sequence: input the segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group, wherein the audio recognition model in the audio recognition model group corresponds to the segmented audio recognition result in the segmented audio recognition result group, and each audio recognition model in the audio recognition model group is different; according to the corresponding audio segmented audio submitted by the user, The method comprises the following steps: according to the selection information of the recognition result group, selecting at least one segmented audio recognition result from the segmented audio recognition result group as the target audio recognition candidate result group; determining whether the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1; the determination unit 203 is configured to determine the audio recognition model corresponding to the target audio recognition candidate result group as the target audio recognition model in response to determining that the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1; and the input unit 204 is configured to input the remaining segmented audio of the target audio into the target audio recognition model to obtain the target audio recognition result.

[0061] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units included therein, and will not be described in detail here.

[0062] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 300 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0063] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0064] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0065] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0066] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0067] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0068] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to detecting an audio recognition selection operation acting on the target audio, performs orderly segmentation processing on the above-mentioned target audio to generate a segmented audio sequence; for each segmented audio in the above-mentioned segmented audio sequence, performs the following processing steps: inputting the above-mentioned segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group, wherein the audio recognition model in the above-mentioned audio recognition model group corresponds to the segmented audio recognition result in the above-mentioned segmented audio recognition result group, and each audio recognition in the above-mentioned audio recognition model group corresponds to the segmented audio recognition result in the above-mentioned segmented audio recognition result group. The different models are different; according to the various selection information corresponding to the above-mentioned segmented audio recognition result group submitted by the user, at least one segmented audio recognition result is selected from the above-mentioned segmented audio recognition result group as the target audio recognition candidate result group; it is determined whether the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1; in response to determining that the number of target audio recognition candidate results included in the above-mentioned target audio recognition candidate result group is 1, the audio recognition model corresponding to the above-mentioned target audio recognition candidate result group is determined as the target audio recognition model; the remaining segmented audio of the above-mentioned target audio is input into the above-mentioned target audio recognition model to obtain the target audio recognition result.

[0069] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0070] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0071] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a segmentation unit, a segmentation audio recognition unit, a determination unit, and an input unit. The names of these units do not constitute limitations on the units themselves in certain circumstances, for example, the segmentation unit may also be described as "a unit that, in response to detecting an audio recognition selection operation acting on a target audio, performs orderly segmentation processing on the target audio to generate a segmented audio sequence".

[0072] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0073] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A multi-model based audio recognition method, include: In response to detecting an audio recognition selection operation acting on a target audio, performing an orderly segmentation process on the target audio to generate a segmented audio sequence; For each segmented audio in the segmented audio sequence, the following processing steps are performed: Inputting the segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group, wherein the audio recognition models in the audio recognition model group correspond to the segmented audio recognition results in the segmented audio recognition result group, and each audio recognition model in the audio recognition model group is different; According to each selection information corresponding to the segmented audio recognition result group submitted by the user, selecting at least one segmented audio recognition result from the segmented audio recognition result group as a target audio recognition candidate result group; Determine whether the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1; In response to determining that the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1, determining the audio recognition model corresponding to the target audio recognition candidate result group as the target audio recognition model; The remaining segmented audio of the target audio is input into the target audio recognition model to obtain a target audio recognition result.

2. The method according to claim 1, in, The method further comprises: In response to determining that the number of target audio recognition candidate results included in the target audio recognition candidate result group is greater than 1, the respective audio recognition models corresponding to the target audio recognition candidate result group are used as the updated audio recognition model group, and the segmented audio group from which the segmented audio is removed is used as the updated segmented audio group, and the processing steps are performed again.

3. The method according to claim 1, in, Before the target audio is sequentially segmented to generate a segmented audio sequence, the method further includes: Receive audio to be recognized submitted by the user; The audio segments that meet the blank audio condition in the audio to be recognized are removed to update the audio to be recognized, and the updated audio to be recognized is used as the target audio, wherein the blank audio condition is that the audio segment does not contain the target voice.

4. The method according to claim 1, in, The step of sequentially segmenting the target audio to generate a segmented audio sequence includes: Determining the duration of the target audio; In response to the duration being greater than or equal to a preset duration, the target audio is sequentially segmented according to a first preset segmentation duration to generate a segmented audio sequence; In response to the duration being less than the preset duration, the target audio is segmented in an orderly manner according to a second preset segmentation duration to generate a segmented audio sequence, wherein the second preset segmentation duration is less than the first preset segmentation duration.

5. An audio recognition device based on multiple models, include: A segmentation unit, configured to, in response to detecting an audio recognition selection operation acting on a target audio, perform an orderly segmentation process on the target audio to generate a segmented audio sequence; The segmented audio recognition unit is configured to perform the following processing steps for each segmented audio in the segmented audio sequence: input the segmented audio into a pre-trained audio recognition model group to obtain a segmented audio recognition result group, wherein the audio recognition models in the audio recognition model group correspond to the segmented audio recognition results in the segmented audio recognition result group, and each audio recognition model in the audio recognition model group is different; select at least one segmented audio recognition result from the segmented audio recognition result group as a target audio recognition candidate result group according to each selection information corresponding to the segmented audio recognition result group submitted by a user; determine whether the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1; a determining unit configured to, in response to determining that the number of target audio recognition candidate results included in the target audio recognition candidate result group is 1, determine the audio recognition model corresponding to the target audio recognition candidate result group as the target audio recognition model; The input unit is configured to input the remaining segmented audio of the target audio into the target audio recognition model to obtain a target audio recognition result.

6. An electronic device, include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, in, When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images